{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Logistic Regression in R A Classification Technique to Predict Credit Card Default\n",
    "\n",
    "## INTRODUCTION\n",
    "\n",
    "Logistic regression is one of the statistical techniques in machine learning used to form prediction models. It is one of the most popular classification algorithms mostly used for binary classification problems (problems with two class values, however, some variants may deal with multiple classes as well). It's used for various research and industrial problems. Therefore, it is essential to have a good grasp on logistic regression algorithm. This tutorial is a sneak peek from many of Data Science Dojo’s hands-on exercises from their 5-day [data science bootcamp](https://datasciencedojo.com/data-science-bootcamp/), you will learn how logistic regression fits a dataset to make predictions, as well as when and why to use it.\n",
    "\n",
    "In short, Logistic Regression is used when the dependent variable(target) is categorical. For example: <br/>\n",
    "* To predict whether an email is spam (1) or not spam (0) <br/>\n",
    "* Whether the tumor is malignant (1) or not (0) <br/>\n",
    "\n",
    "It is named as ‘Logistic Regression’, because it’s underlying technique is quite the same as Linear Regression. There are structural differences in how linear and logistic regression operate. Therefore, linear regression isn't suitable to be used for classification problems. This [link](https://stats.stackexchange.com/questions/22381/why-not-approach-classification-through-regression) answers in details that why linear regression isn't the right approach for classification.\n",
    "\n",
    "Its name is derived from one of the core function behind its implementation called the logistic function or the sigmoid function. It’s an S-shaped curve that can take any real-valued number and map it into a value between 0 and 1, but never exactly at those limits.\n",
    "\n",
    "![Logistic Function](https://cdn-images-1.medium.com/max/1500/1*sR6PgnEB5HmuRuz62DfWzQ.png)\n",
    "\n",
    "The hypothesis function of logistic regression can be seen below where the function g(z) is also shown.\n",
    "\n",
    "![Logistic Regression Hypothesis](https://cdn-images-1.medium.com/max/2000/1*o4Dy1w4n2kDOLA8UEwGC9g.png)\n",
    "\n",
    "The hypothesis for logistic regression now becomes:\n",
    "\n",
    "![](https://cdn-images-1.medium.com/max/1000/1*L9a6phB1ZzjRhb-VI3W1YQ.png)\n",
    "\n",
    "![Hypothesis function](https://cdn-images-1.medium.com/max/1000/1*jStEeKa6l6KgQxbS8iGzrw.png)\n",
    "\n",
    "\n",
    "Here θ (theta) is a vector of paramters that our model will calculate to fit our classifier.\n",
    "\n",
    "After calculations from the above equations, the cost function is now as follows:\n",
    "\n",
    "![](https://cdn-images-1.medium.com/max/1000/1*Capcm4gWSZFZY6Str-wSRQ.png)\n",
    "\n",
    "Here m is the number of training examples. Like  Linear Regression, we will use gradient descent to minimize our cost function and calculate the vector θ (theta). \n",
    "\n",
    "----\n",
    "\n",
    "This tutorial will follow the format below to provide you hands-on practice with Logistic Regression:\n",
    "\n",
    "1. Importing Libraries\n",
    "2. Importing Datasets\n",
    "3. Exploratory Data Analysis\n",
    "4. Feature Engineering\n",
    "5. Pre-processing\n",
    "6. Model Development\n",
    "7. Prediction\n",
    "8. Evaluation\n",
    "\n",
    "----\n",
    "\n",
    "## THE SCENARIO\n",
    "\n",
    "In this tutorial, we will be working with [Default of Credit Card Clients Data Set](https://code.datasciencedojo.com/datasciencedojo/datasets/tree/master/Default%20of%20Credit%20Card%20Clients). This data set has 30000 rows and 24 columns. The data set could be used to estimate the probability of default payment by credit card client using the data provided. These attributes are related to various details about a customer, his past payment information and bill statements. It is hosted in Data Science Dojo's repository. \n",
    "\n",
    "Think of yourself as a lead data scientist employed at a large bank. You have been assigned to predict whether a particular customer will default payment next month or not. The result is a an extremely valuable piece of information for the bank to take decisions regarding offering credit to its customer and could massively affect the bank's revenue. Therefore, your task is very critical. You will learn to use logistic regression to solve this problem.\n",
    "\n",
    "The dataset is a tricky one as it has a mix of categorical and continuous variables. Moreover, You will also get a chance to practice these concepts through short assignments given at the end of a few sub-module. Feel free to change the parameters in the given methods once you have been through the entire notebook.\n",
    "\n",
    "\n",
    "-----\n",
    "\n",
    "## 1) Importing Libraries\n",
    "\n",
    "We'll begin by importing our dependencies that we require. The following dependencies are popularly used for data wrangling operations and visualizations. We would encourage you to have a look at their documentations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "library(knitr)\n",
    "library(tidyverse)\n",
    "library(ggplot2)\n",
    "library(mice)\n",
    "library(lattice)\n",
    "library(reshape2)\n",
    "library(RCurl, quietly = TRUE)\n",
    "library(DataExplorer)\n",
    "options(warn=-1)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2) Importing Datasets\n",
    "\n",
    "The dataset is available at Data Science Dojo's repository in the following [link](https://code.datasciencedojo.com/datasciencedojo/datasets/tree/master/Default%20of%20Credit%20Card%20Clients). We'll use head method to view the first few rows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<thead><tr><th scope=col>X</th><th scope=col>X1</th><th scope=col>X2</th><th scope=col>X3</th><th scope=col>X4</th><th scope=col>X5</th><th scope=col>X6</th><th scope=col>X7</th><th scope=col>X8</th><th scope=col>X9</th><th scope=col>...</th><th scope=col>X15</th><th scope=col>X16</th><th scope=col>X17</th><th scope=col>X18</th><th scope=col>X19</th><th scope=col>X20</th><th scope=col>X21</th><th scope=col>X22</th><th scope=col>X23</th><th scope=col>Y</th></tr></thead>\n",
       "<tbody>\n",
       "\t<tr><td>ID                        </td><td>LIMIT_BAL                 </td><td>SEX                       </td><td>EDUCATION                 </td><td>MARRIAGE                  </td><td>AGE                       </td><td>PAY_0                     </td><td>PAY_2                     </td><td>PAY_3                     </td><td>PAY_4                     </td><td>...                       </td><td>BILL_AMT4                 </td><td>BILL_AMT5                 </td><td>BILL_AMT6                 </td><td>PAY_AMT1                  </td><td>PAY_AMT2                  </td><td>PAY_AMT3                  </td><td>PAY_AMT4                  </td><td>PAY_AMT5                  </td><td>PAY_AMT6                  </td><td>default payment next month</td></tr>\n",
       "\t<tr><td>1                         </td><td>20000                     </td><td>2                         </td><td>2                         </td><td>1                         </td><td>24                        </td><td>2                         </td><td>2                         </td><td>-1                        </td><td>-1                        </td><td>...                       </td><td>0                         </td><td>0                         </td><td>0                         </td><td>0                         </td><td>689                       </td><td>0                         </td><td>0                         </td><td>0                         </td><td>0                         </td><td>1                         </td></tr>\n",
       "\t<tr><td>2                         </td><td>120000                    </td><td>2                         </td><td>2                         </td><td>2                         </td><td>26                        </td><td>-1                        </td><td>2                         </td><td>0                         </td><td>0                         </td><td>...                       </td><td>3272                      </td><td>3455                      </td><td>3261                      </td><td>0                         </td><td>1000                      </td><td>1000                      </td><td>1000                      </td><td>0                         </td><td>2000                      </td><td>1                         </td></tr>\n",
       "\t<tr><td>3                         </td><td>90000                     </td><td>2                         </td><td>2                         </td><td>2                         </td><td>34                        </td><td>0                         </td><td>0                         </td><td>0                         </td><td>0                         </td><td>...                       </td><td>14331                     </td><td>14948                     </td><td>15549                     </td><td>1518                      </td><td>1500                      </td><td>1000                      </td><td>1000                      </td><td>1000                      </td><td>5000                      </td><td>0                         </td></tr>\n",
       "\t<tr><td>4                         </td><td>50000                     </td><td>2                         </td><td>2                         </td><td>1                         </td><td>37                        </td><td>0                         </td><td>0                         </td><td>0                         </td><td>0                         </td><td>...                       </td><td>28314                     </td><td>28959                     </td><td>29547                     </td><td>2000                      </td><td>2019                      </td><td>1200                      </td><td>1100                      </td><td>1069                      </td><td>1000                      </td><td>0                         </td></tr>\n",
       "\t<tr><td>5                         </td><td>50000                     </td><td>1                         </td><td>2                         </td><td>1                         </td><td>57                        </td><td>-1                        </td><td>0                         </td><td>-1                        </td><td>0                         </td><td>...                       </td><td>20940                     </td><td>19146                     </td><td>19131                     </td><td>2000                      </td><td>36681                     </td><td>10000                     </td><td>9000                      </td><td>689                       </td><td>679                       </td><td>0                         </td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "\\begin{tabular}{r|lllllllllllllllllllllllll}\n",
       " X & X1 & X2 & X3 & X4 & X5 & X6 & X7 & X8 & X9 & ... & X15 & X16 & X17 & X18 & X19 & X20 & X21 & X22 & X23 & Y\\\\\n",
       "\\hline\n",
       "\t ID                         & LIMIT\\_BAL                & SEX                        & EDUCATION                  & MARRIAGE                   & AGE                        & PAY\\_0                    & PAY\\_2                    & PAY\\_3                    & PAY\\_4                    & ...                        & BILL\\_AMT4                & BILL\\_AMT5                & BILL\\_AMT6                & PAY\\_AMT1                 & PAY\\_AMT2                 & PAY\\_AMT3                 & PAY\\_AMT4                 & PAY\\_AMT5                 & PAY\\_AMT6                 & default payment next month\\\\\n",
       "\t 1                          & 20000                      & 2                          & 2                          & 1                          & 24                         & 2                          & 2                          & -1                         & -1                         & ...                        & 0                          & 0                          & 0                          & 0                          & 689                        & 0                          & 0                          & 0                          & 0                          & 1                         \\\\\n",
       "\t 2                          & 120000                     & 2                          & 2                          & 2                          & 26                         & -1                         & 2                          & 0                          & 0                          & ...                        & 3272                       & 3455                       & 3261                       & 0                          & 1000                       & 1000                       & 1000                       & 0                          & 2000                       & 1                         \\\\\n",
       "\t 3                          & 90000                      & 2                          & 2                          & 2                          & 34                         & 0                          & 0                          & 0                          & 0                          & ...                        & 14331                      & 14948                      & 15549                      & 1518                       & 1500                       & 1000                       & 1000                       & 1000                       & 5000                       & 0                         \\\\\n",
       "\t 4                          & 50000                      & 2                          & 2                          & 1                          & 37                         & 0                          & 0                          & 0                          & 0                          & ...                        & 28314                      & 28959                      & 29547                      & 2000                       & 2019                       & 1200                       & 1100                       & 1069                       & 1000                       & 0                         \\\\\n",
       "\t 5                          & 50000                      & 1                          & 2                          & 1                          & 57                         & -1                         & 0                          & -1                         & 0                          & ...                        & 20940                      & 19146                      & 19131                      & 2000                       & 36681                      & 10000                      & 9000                       & 689                        & 679                        & 0                         \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "| X | X1 | X2 | X3 | X4 | X5 | X6 | X7 | X8 | X9 | ... | X15 | X16 | X17 | X18 | X19 | X20 | X21 | X22 | X23 | Y |\n",
       "|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|\n",
       "| ID                         | LIMIT_BAL                  | SEX                        | EDUCATION                  | MARRIAGE                   | AGE                        | PAY_0                      | PAY_2                      | PAY_3                      | PAY_4                      | ...                        | BILL_AMT4                  | BILL_AMT5                  | BILL_AMT6                  | PAY_AMT1                   | PAY_AMT2                   | PAY_AMT3                   | PAY_AMT4                   | PAY_AMT5                   | PAY_AMT6                   | default payment next month |\n",
       "| 1                          | 20000                      | 2                          | 2                          | 1                          | 24                         | 2                          | 2                          | -1                         | -1                         | ...                        | 0                          | 0                          | 0                          | 0                          | 689                        | 0                          | 0                          | 0                          | 0                          | 1                          |\n",
       "| 2                          | 120000                     | 2                          | 2                          | 2                          | 26                         | -1                         | 2                          | 0                          | 0                          | ...                        | 3272                       | 3455                       | 3261                       | 0                          | 1000                       | 1000                       | 1000                       | 0                          | 2000                       | 1                          |\n",
       "| 3                          | 90000                      | 2                          | 2                          | 2                          | 34                         | 0                          | 0                          | 0                          | 0                          | ...                        | 14331                      | 14948                      | 15549                      | 1518                       | 1500                       | 1000                       | 1000                       | 1000                       | 5000                       | 0                          |\n",
       "| 4                          | 50000                      | 2                          | 2                          | 1                          | 37                         | 0                          | 0                          | 0                          | 0                          | ...                        | 28314                      | 28959                      | 29547                      | 2000                       | 2019                       | 1200                       | 1100                       | 1069                       | 1000                       | 0                          |\n",
       "| 5                          | 50000                      | 1                          | 2                          | 1                          | 57                         | -1                         | 0                          | -1                         | 0                          | ...                        | 20940                      | 19146                      | 19131                      | 2000                       | 36681                      | 10000                      | 9000                       | 689                        | 679                        | 0                          |\n",
       "\n"
      ],
      "text/plain": [
       "  X  X1        X2  X3        X4       X5  X6    X7    X8    X9    ... X15      \n",
       "1 ID LIMIT_BAL SEX EDUCATION MARRIAGE AGE PAY_0 PAY_2 PAY_3 PAY_4 ... BILL_AMT4\n",
       "2 1  20000     2   2         1        24  2     2     -1    -1    ... 0        \n",
       "3 2  120000    2   2         2        26  -1    2     0     0     ... 3272     \n",
       "4 3  90000     2   2         2        34  0     0     0     0     ... 14331    \n",
       "5 4  50000     2   2         1        37  0     0     0     0     ... 28314    \n",
       "6 5  50000     1   2         1        57  -1    0     -1    0     ... 20940    \n",
       "  X16       X17       X18      X19      X20      X21      X22      X23     \n",
       "1 BILL_AMT5 BILL_AMT6 PAY_AMT1 PAY_AMT2 PAY_AMT3 PAY_AMT4 PAY_AMT5 PAY_AMT6\n",
       "2 0         0         0        689      0        0        0        0       \n",
       "3 3455      3261      0        1000     1000     1000     0        2000    \n",
       "4 14948     15549     1518     1500     1000     1000     1000     5000    \n",
       "5 28959     29547     2000     2019     1200     1100     1069     1000    \n",
       "6 19146     19131     2000     36681    10000    9000     689      679     \n",
       "  Y                         \n",
       "1 default payment next month\n",
       "2 1                         \n",
       "3 1                         \n",
       "4 0                         \n",
       "5 0                         \n",
       "6 0                         "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "## Need to fetch the excel file\n",
    "\n",
    "path <- \"https://code.datasciencedojo.com/datasciencedojo/datasets/raw/master/Default%20of%20Credit%20Card%20Clients/default%20of%20credit%20card%20clients.csv\"\n",
    "data <- read.csv(file = path, header = TRUE)\n",
    "head(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since the header names are in the first row of the dataset, we'll use the code below to first assign the headers to be the one from the first row and then delete the first row from the dataset. This way we will get our desired form."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<thead><tr><th></th><th scope=col>ID</th><th scope=col>LIMIT_BAL</th><th scope=col>SEX</th><th scope=col>EDUCATION</th><th scope=col>MARRIAGE</th><th scope=col>AGE</th><th scope=col>PAY_0</th><th scope=col>PAY_2</th><th scope=col>PAY_3</th><th scope=col>PAY_4</th><th scope=col>...</th><th scope=col>BILL_AMT4</th><th scope=col>BILL_AMT5</th><th scope=col>BILL_AMT6</th><th scope=col>PAY_AMT1</th><th scope=col>PAY_AMT2</th><th scope=col>PAY_AMT3</th><th scope=col>PAY_AMT4</th><th scope=col>PAY_AMT5</th><th scope=col>PAY_AMT6</th><th scope=col>default payment next month</th></tr></thead>\n",
       "<tbody>\n",
       "\t<tr><th scope=row>2</th><td>1     </td><td>20000 </td><td>2     </td><td>2     </td><td>1     </td><td>24    </td><td>2     </td><td>2     </td><td>-1    </td><td>-1    </td><td>...   </td><td>0     </td><td>0     </td><td>0     </td><td>0     </td><td>689   </td><td>0     </td><td>0     </td><td>0     </td><td>0     </td><td>1     </td></tr>\n",
       "\t<tr><th scope=row>3</th><td>2     </td><td>120000</td><td>2     </td><td>2     </td><td>2     </td><td>26    </td><td>-1    </td><td>2     </td><td>0     </td><td>0     </td><td>...   </td><td>3272  </td><td>3455  </td><td>3261  </td><td>0     </td><td>1000  </td><td>1000  </td><td>1000  </td><td>0     </td><td>2000  </td><td>1     </td></tr>\n",
       "\t<tr><th scope=row>4</th><td>3     </td><td>90000 </td><td>2     </td><td>2     </td><td>2     </td><td>34    </td><td>0     </td><td>0     </td><td>0     </td><td>0     </td><td>...   </td><td>14331 </td><td>14948 </td><td>15549 </td><td>1518  </td><td>1500  </td><td>1000  </td><td>1000  </td><td>1000  </td><td>5000  </td><td>0     </td></tr>\n",
       "\t<tr><th scope=row>5</th><td>4     </td><td>50000 </td><td>2     </td><td>2     </td><td>1     </td><td>37    </td><td>0     </td><td>0     </td><td>0     </td><td>0     </td><td>...   </td><td>28314 </td><td>28959 </td><td>29547 </td><td>2000  </td><td>2019  </td><td>1200  </td><td>1100  </td><td>1069  </td><td>1000  </td><td>0     </td></tr>\n",
       "\t<tr><th scope=row>6</th><td>5     </td><td>50000 </td><td>1     </td><td>2     </td><td>1     </td><td>57    </td><td>-1    </td><td>0     </td><td>-1    </td><td>0     </td><td>...   </td><td>20940 </td><td>19146 </td><td>19131 </td><td>2000  </td><td>36681 </td><td>10000 </td><td>9000  </td><td>689   </td><td>679   </td><td>0     </td></tr>\n",
       "\t<tr><th scope=row>7</th><td>6     </td><td>50000 </td><td>1     </td><td>1     </td><td>2     </td><td>37    </td><td>0     </td><td>0     </td><td>0     </td><td>0     </td><td>...   </td><td>19394 </td><td>19619 </td><td>20024 </td><td>2500  </td><td>1815  </td><td>657   </td><td>1000  </td><td>1000  </td><td>800   </td><td>0     </td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "\\begin{tabular}{r|lllllllllllllllllllllllll}\n",
       "  & ID & LIMIT\\_BAL & SEX & EDUCATION & MARRIAGE & AGE & PAY\\_0 & PAY\\_2 & PAY\\_3 & PAY\\_4 & ... & BILL\\_AMT4 & BILL\\_AMT5 & BILL\\_AMT6 & PAY\\_AMT1 & PAY\\_AMT2 & PAY\\_AMT3 & PAY\\_AMT4 & PAY\\_AMT5 & PAY\\_AMT6 & default payment next month\\\\\n",
       "\\hline\n",
       "\t2 & 1      & 20000  & 2      & 2      & 1      & 24     & 2      & 2      & -1     & -1     & ...    & 0      & 0      & 0      & 0      & 689    & 0      & 0      & 0      & 0      & 1     \\\\\n",
       "\t3 & 2      & 120000 & 2      & 2      & 2      & 26     & -1     & 2      & 0      & 0      & ...    & 3272   & 3455   & 3261   & 0      & 1000   & 1000   & 1000   & 0      & 2000   & 1     \\\\\n",
       "\t4 & 3      & 90000  & 2      & 2      & 2      & 34     & 0      & 0      & 0      & 0      & ...    & 14331  & 14948  & 15549  & 1518   & 1500   & 1000   & 1000   & 1000   & 5000   & 0     \\\\\n",
       "\t5 & 4      & 50000  & 2      & 2      & 1      & 37     & 0      & 0      & 0      & 0      & ...    & 28314  & 28959  & 29547  & 2000   & 2019   & 1200   & 1100   & 1069   & 1000   & 0     \\\\\n",
       "\t6 & 5      & 50000  & 1      & 2      & 1      & 57     & -1     & 0      & -1     & 0      & ...    & 20940  & 19146  & 19131  & 2000   & 36681  & 10000  & 9000   & 689    & 679    & 0     \\\\\n",
       "\t7 & 6      & 50000  & 1      & 1      & 2      & 37     & 0      & 0      & 0      & 0      & ...    & 19394  & 19619  & 20024  & 2500   & 1815   & 657    & 1000   & 1000   & 800    & 0     \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "| <!--/--> | ID | LIMIT_BAL | SEX | EDUCATION | MARRIAGE | AGE | PAY_0 | PAY_2 | PAY_3 | PAY_4 | ... | BILL_AMT4 | BILL_AMT5 | BILL_AMT6 | PAY_AMT1 | PAY_AMT2 | PAY_AMT3 | PAY_AMT4 | PAY_AMT5 | PAY_AMT6 | default payment next month |\n",
       "|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|\n",
       "| 2 | 1      | 20000  | 2      | 2      | 1      | 24     | 2      | 2      | -1     | -1     | ...    | 0      | 0      | 0      | 0      | 689    | 0      | 0      | 0      | 0      | 1      |\n",
       "| 3 | 2      | 120000 | 2      | 2      | 2      | 26     | -1     | 2      | 0      | 0      | ...    | 3272   | 3455   | 3261   | 0      | 1000   | 1000   | 1000   | 0      | 2000   | 1      |\n",
       "| 4 | 3      | 90000  | 2      | 2      | 2      | 34     | 0      | 0      | 0      | 0      | ...    | 14331  | 14948  | 15549  | 1518   | 1500   | 1000   | 1000   | 1000   | 5000   | 0      |\n",
       "| 5 | 4      | 50000  | 2      | 2      | 1      | 37     | 0      | 0      | 0      | 0      | ...    | 28314  | 28959  | 29547  | 2000   | 2019   | 1200   | 1100   | 1069   | 1000   | 0      |\n",
       "| 6 | 5      | 50000  | 1      | 2      | 1      | 57     | -1     | 0      | -1     | 0      | ...    | 20940  | 19146  | 19131  | 2000   | 36681  | 10000  | 9000   | 689    | 679    | 0      |\n",
       "| 7 | 6      | 50000  | 1      | 1      | 2      | 37     | 0      | 0      | 0      | 0      | ...    | 19394  | 19619  | 20024  | 2500   | 1815   | 657    | 1000   | 1000   | 800    | 0      |\n",
       "\n"
      ],
      "text/plain": [
       "  ID LIMIT_BAL SEX EDUCATION MARRIAGE AGE PAY_0 PAY_2 PAY_3 PAY_4 ... BILL_AMT4\n",
       "2 1  20000     2   2         1        24  2     2     -1    -1    ... 0        \n",
       "3 2  120000    2   2         2        26  -1    2     0     0     ... 3272     \n",
       "4 3  90000     2   2         2        34  0     0     0     0     ... 14331    \n",
       "5 4  50000     2   2         1        37  0     0     0     0     ... 28314    \n",
       "6 5  50000     1   2         1        57  -1    0     -1    0     ... 20940    \n",
       "7 6  50000     1   1         2        37  0     0     0     0     ... 19394    \n",
       "  BILL_AMT5 BILL_AMT6 PAY_AMT1 PAY_AMT2 PAY_AMT3 PAY_AMT4 PAY_AMT5 PAY_AMT6\n",
       "2 0         0         0        689      0        0        0        0       \n",
       "3 3455      3261      0        1000     1000     1000     0        2000    \n",
       "4 14948     15549     1518     1500     1000     1000     1000     5000    \n",
       "5 28959     29547     2000     2019     1200     1100     1069     1000    \n",
       "6 19146     19131     2000     36681    10000    9000     689      679     \n",
       "7 19619     20024     2500     1815     657      1000     1000     800     \n",
       "  default payment next month\n",
       "2 1                         \n",
       "3 1                         \n",
       "4 0                         \n",
       "5 0                         \n",
       "6 0                         \n",
       "7 0                         "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "colnames(data) <- as.character(unlist(data[1,]))\n",
    "data = data[-1, ]\n",
    "head(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To avoid any complications ahead, we'll rename our target variable \"default payment next month\" to a name without spaces using the code below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<thead><tr><th></th><th scope=col>ID</th><th scope=col>LIMIT_BAL</th><th scope=col>SEX</th><th scope=col>EDUCATION</th><th scope=col>MARRIAGE</th><th scope=col>AGE</th><th scope=col>PAY_0</th><th scope=col>PAY_2</th><th scope=col>PAY_3</th><th scope=col>PAY_4</th><th scope=col>...</th><th scope=col>BILL_AMT4</th><th scope=col>BILL_AMT5</th><th scope=col>BILL_AMT6</th><th scope=col>PAY_AMT1</th><th scope=col>PAY_AMT2</th><th scope=col>PAY_AMT3</th><th scope=col>PAY_AMT4</th><th scope=col>PAY_AMT5</th><th scope=col>PAY_AMT6</th><th scope=col>default_payment</th></tr></thead>\n",
       "<tbody>\n",
       "\t<tr><th scope=row>2</th><td>1     </td><td>20000 </td><td>2     </td><td>2     </td><td>1     </td><td>24    </td><td>2     </td><td>2     </td><td>-1    </td><td>-1    </td><td>...   </td><td>0     </td><td>0     </td><td>0     </td><td>0     </td><td>689   </td><td>0     </td><td>0     </td><td>0     </td><td>0     </td><td>1     </td></tr>\n",
       "\t<tr><th scope=row>3</th><td>2     </td><td>120000</td><td>2     </td><td>2     </td><td>2     </td><td>26    </td><td>-1    </td><td>2     </td><td>0     </td><td>0     </td><td>...   </td><td>3272  </td><td>3455  </td><td>3261  </td><td>0     </td><td>1000  </td><td>1000  </td><td>1000  </td><td>0     </td><td>2000  </td><td>1     </td></tr>\n",
       "\t<tr><th scope=row>4</th><td>3     </td><td>90000 </td><td>2     </td><td>2     </td><td>2     </td><td>34    </td><td>0     </td><td>0     </td><td>0     </td><td>0     </td><td>...   </td><td>14331 </td><td>14948 </td><td>15549 </td><td>1518  </td><td>1500  </td><td>1000  </td><td>1000  </td><td>1000  </td><td>5000  </td><td>0     </td></tr>\n",
       "\t<tr><th scope=row>5</th><td>4     </td><td>50000 </td><td>2     </td><td>2     </td><td>1     </td><td>37    </td><td>0     </td><td>0     </td><td>0     </td><td>0     </td><td>...   </td><td>28314 </td><td>28959 </td><td>29547 </td><td>2000  </td><td>2019  </td><td>1200  </td><td>1100  </td><td>1069  </td><td>1000  </td><td>0     </td></tr>\n",
       "\t<tr><th scope=row>6</th><td>5     </td><td>50000 </td><td>1     </td><td>2     </td><td>1     </td><td>57    </td><td>-1    </td><td>0     </td><td>-1    </td><td>0     </td><td>...   </td><td>20940 </td><td>19146 </td><td>19131 </td><td>2000  </td><td>36681 </td><td>10000 </td><td>9000  </td><td>689   </td><td>679   </td><td>0     </td></tr>\n",
       "\t<tr><th scope=row>7</th><td>6     </td><td>50000 </td><td>1     </td><td>1     </td><td>2     </td><td>37    </td><td>0     </td><td>0     </td><td>0     </td><td>0     </td><td>...   </td><td>19394 </td><td>19619 </td><td>20024 </td><td>2500  </td><td>1815  </td><td>657   </td><td>1000  </td><td>1000  </td><td>800   </td><td>0     </td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "\\begin{tabular}{r|lllllllllllllllllllllllll}\n",
       "  & ID & LIMIT\\_BAL & SEX & EDUCATION & MARRIAGE & AGE & PAY\\_0 & PAY\\_2 & PAY\\_3 & PAY\\_4 & ... & BILL\\_AMT4 & BILL\\_AMT5 & BILL\\_AMT6 & PAY\\_AMT1 & PAY\\_AMT2 & PAY\\_AMT3 & PAY\\_AMT4 & PAY\\_AMT5 & PAY\\_AMT6 & default\\_payment\\\\\n",
       "\\hline\n",
       "\t2 & 1      & 20000  & 2      & 2      & 1      & 24     & 2      & 2      & -1     & -1     & ...    & 0      & 0      & 0      & 0      & 689    & 0      & 0      & 0      & 0      & 1     \\\\\n",
       "\t3 & 2      & 120000 & 2      & 2      & 2      & 26     & -1     & 2      & 0      & 0      & ...    & 3272   & 3455   & 3261   & 0      & 1000   & 1000   & 1000   & 0      & 2000   & 1     \\\\\n",
       "\t4 & 3      & 90000  & 2      & 2      & 2      & 34     & 0      & 0      & 0      & 0      & ...    & 14331  & 14948  & 15549  & 1518   & 1500   & 1000   & 1000   & 1000   & 5000   & 0     \\\\\n",
       "\t5 & 4      & 50000  & 2      & 2      & 1      & 37     & 0      & 0      & 0      & 0      & ...    & 28314  & 28959  & 29547  & 2000   & 2019   & 1200   & 1100   & 1069   & 1000   & 0     \\\\\n",
       "\t6 & 5      & 50000  & 1      & 2      & 1      & 57     & -1     & 0      & -1     & 0      & ...    & 20940  & 19146  & 19131  & 2000   & 36681  & 10000  & 9000   & 689    & 679    & 0     \\\\\n",
       "\t7 & 6      & 50000  & 1      & 1      & 2      & 37     & 0      & 0      & 0      & 0      & ...    & 19394  & 19619  & 20024  & 2500   & 1815   & 657    & 1000   & 1000   & 800    & 0     \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "| <!--/--> | ID | LIMIT_BAL | SEX | EDUCATION | MARRIAGE | AGE | PAY_0 | PAY_2 | PAY_3 | PAY_4 | ... | BILL_AMT4 | BILL_AMT5 | BILL_AMT6 | PAY_AMT1 | PAY_AMT2 | PAY_AMT3 | PAY_AMT4 | PAY_AMT5 | PAY_AMT6 | default_payment |\n",
       "|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|\n",
       "| 2 | 1      | 20000  | 2      | 2      | 1      | 24     | 2      | 2      | -1     | -1     | ...    | 0      | 0      | 0      | 0      | 689    | 0      | 0      | 0      | 0      | 1      |\n",
       "| 3 | 2      | 120000 | 2      | 2      | 2      | 26     | -1     | 2      | 0      | 0      | ...    | 3272   | 3455   | 3261   | 0      | 1000   | 1000   | 1000   | 0      | 2000   | 1      |\n",
       "| 4 | 3      | 90000  | 2      | 2      | 2      | 34     | 0      | 0      | 0      | 0      | ...    | 14331  | 14948  | 15549  | 1518   | 1500   | 1000   | 1000   | 1000   | 5000   | 0      |\n",
       "| 5 | 4      | 50000  | 2      | 2      | 1      | 37     | 0      | 0      | 0      | 0      | ...    | 28314  | 28959  | 29547  | 2000   | 2019   | 1200   | 1100   | 1069   | 1000   | 0      |\n",
       "| 6 | 5      | 50000  | 1      | 2      | 1      | 57     | -1     | 0      | -1     | 0      | ...    | 20940  | 19146  | 19131  | 2000   | 36681  | 10000  | 9000   | 689    | 679    | 0      |\n",
       "| 7 | 6      | 50000  | 1      | 1      | 2      | 37     | 0      | 0      | 0      | 0      | ...    | 19394  | 19619  | 20024  | 2500   | 1815   | 657    | 1000   | 1000   | 800    | 0      |\n",
       "\n"
      ],
      "text/plain": [
       "  ID LIMIT_BAL SEX EDUCATION MARRIAGE AGE PAY_0 PAY_2 PAY_3 PAY_4 ... BILL_AMT4\n",
       "2 1  20000     2   2         1        24  2     2     -1    -1    ... 0        \n",
       "3 2  120000    2   2         2        26  -1    2     0     0     ... 3272     \n",
       "4 3  90000     2   2         2        34  0     0     0     0     ... 14331    \n",
       "5 4  50000     2   2         1        37  0     0     0     0     ... 28314    \n",
       "6 5  50000     1   2         1        57  -1    0     -1    0     ... 20940    \n",
       "7 6  50000     1   1         2        37  0     0     0     0     ... 19394    \n",
       "  BILL_AMT5 BILL_AMT6 PAY_AMT1 PAY_AMT2 PAY_AMT3 PAY_AMT4 PAY_AMT5 PAY_AMT6\n",
       "2 0         0         0        689      0        0        0        0       \n",
       "3 3455      3261      0        1000     1000     1000     0        2000    \n",
       "4 14948     15549     1518     1500     1000     1000     1000     5000    \n",
       "5 28959     29547     2000     2019     1200     1100     1069     1000    \n",
       "6 19146     19131     2000     36681    10000    9000     689      679     \n",
       "7 19619     20024     2500     1815     657      1000     1000     800     \n",
       "  default_payment\n",
       "2 1              \n",
       "3 1              \n",
       "4 0              \n",
       "5 0              \n",
       "6 0              \n",
       "7 0              "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "colnames(data)[colnames(data)==\"default payment next month\"] <- \"default_payment\"\n",
    "head(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3) Exploratory Data Analysis\n",
    "\n",
    "Data Exploration is one of the most significant portions of the machine learning process. Clean data can ensures a notable increase in accuracy of our model. No matter how powerful our model is, it cannot function well unless the data we provide it has been thoroughly processed. This step will briefly take you through this step and assist you to visualize your data, find relation between variables, deal with missing values and outliers and assist in getting some fundamental understanding of each variable we'll use. Moreover, this step will also enable us to figure out the most important attibutes to feed our model and discard those that have no relevance.\n",
    "\n",
    "We will start with using the dim function to print out the dimensionality of our dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<ol class=list-inline>\n",
       "\t<li>30000</li>\n",
       "\t<li>25</li>\n",
       "</ol>\n"
      ],
      "text/latex": [
       "\\begin{enumerate*}\n",
       "\\item 30000\n",
       "\\item 25\n",
       "\\end{enumerate*}\n"
      ],
      "text/markdown": [
       "1. 30000\n",
       "2. 25\n",
       "\n",
       "\n"
      ],
      "text/plain": [
       "[1] 30000    25"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "dim(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The str method will allows us to know the data type of each variable. We'll transform it to numeric data type since it'll be more handy to use for our functions ahead."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "'data.frame':\t30000 obs. of  25 variables:\n",
      " $ ID             : Factor w/ 30001 levels \"1\",\"10\",\"100\",..: 1 11112 22223 23335 24446 25557 26668 27779 28890 2 ...\n",
      " $ LIMIT_BAL      : Factor w/ 82 levels \"10000\",\"100000\",..: 14 5 81 48 48 48 49 2 7 14 ...\n",
      " $ SEX            : Factor w/ 3 levels \"1\",\"2\",\"SEX\": 2 2 2 2 1 1 1 2 2 1 ...\n",
      " $ EDUCATION      : Factor w/ 8 levels \"0\",\"1\",\"2\",\"3\",..: 3 3 3 3 3 2 2 3 4 4 ...\n",
      " $ MARRIAGE       : Factor w/ 5 levels \"0\",\"1\",\"2\",\"3\",..: 2 3 3 2 2 3 3 3 2 3 ...\n",
      " $ AGE            : Factor w/ 57 levels \"21\",\"22\",\"23\",..: 4 6 14 17 37 17 9 3 8 15 ...\n",
      " $ PAY_0          : Factor w/ 12 levels \"-1\",\"-2\",\"0\",..: 5 1 3 3 1 3 3 3 3 2 ...\n",
      " $ PAY_2          : Factor w/ 12 levels \"-1\",\"-2\",\"0\",..: 5 5 3 3 3 3 3 1 3 2 ...\n",
      " $ PAY_3          : Factor w/ 12 levels \"-1\",\"-2\",\"0\",..: 1 3 3 3 1 3 3 1 5 2 ...\n",
      " $ PAY_4          : Factor w/ 12 levels \"-1\",\"-2\",\"0\",..: 1 3 3 3 3 3 3 3 3 2 ...\n",
      " $ PAY_5          : Factor w/ 11 levels \"-1\",\"-2\",\"0\",..: 2 3 3 3 3 3 3 3 3 1 ...\n",
      " $ PAY_6          : Factor w/ 11 levels \"-1\",\"-2\",\"0\",..: 2 4 3 3 3 3 3 1 3 1 ...\n",
      " $ BILL_AMT1      : Factor w/ 22724 levels \"-1\",\"-10\",\"-100\",..: 13345 10030 10924 15026 21268 18423 12835 1993 1518 307 ...\n",
      " $ BILL_AMT2      : Factor w/ 22347 levels \"-1\",\"-10\",\"-100\",..: 11404 5552 3482 15171 16961 17010 13627 12949 3530 348 ...\n",
      " $ BILL_AMT3      : Factor w/ 22027 levels \"-1\",\"-10\",\"-100\",..: 18440 9759 3105 15397 12421 16866 14184 17258 2072 365 ...\n",
      " $ BILL_AMT4      : Factor w/ 21549 levels \"-1\",\"-10\",\"-100\",..: 378 11833 3620 10318 7717 6809 16081 8147 2129 378 ...\n",
      " $ BILL_AMT5      : Factor w/ 21011 levels \"-1\",\"-10\",\"-100\",..: 385 11971 3950 10407 6477 6841 14580 76 1796 2638 ...\n",
      " $ BILL_AMT6      : Factor w/ 20605 levels \"-1\",\"-10\",\"-100\",..: 415 11339 4234 10458 6345 7002 14057 15748 12215 3230 ...\n",
      " $ PAY_AMT1       : Factor w/ 7944 levels \"0\",\"1\",\"10\",\"100\",..: 1 1 1495 2416 2416 3160 5871 4578 4128 1 ...\n",
      " $ PAY_AMT2       : Factor w/ 7900 levels \"0\",\"1\",\"10\",\"100\",..: 6671 5 1477 2536 4508 2142 4778 6189 1 1 ...\n",
      " $ PAY_AMT3       : Factor w/ 7519 levels \"0\",\"1\",\"10\",\"100\",..: 1 5 5 646 6 6163 4292 1 4731 1 ...\n",
      " $ PAY_AMT4       : Factor w/ 6938 levels \"0\",\"1\",\"10\",\"100\",..: 1 5 5 337 6620 5 2077 5286 5 813 ...\n",
      " $ PAY_AMT5       : Factor w/ 6898 levels \"0\",\"1\",\"10\",\"100\",..: 1 1 5 263 5777 5 950 1502 5 408 ...\n",
      " $ PAY_AMT6       : Factor w/ 6940 levels \"0\",\"1\",\"10\",\"100\",..: 1 2003 4751 5 5796 6293 963 1267 5 1 ...\n",
      " $ default_payment: Factor w/ 3 levels \"0\",\"1\",\"default payment next month\": 2 2 1 1 1 1 1 1 1 1 ...\n"
     ]
    }
   ],
   "source": [
    "str(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "data[, 1:25] <- sapply(data[, 1:25], as.character)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have involved an intermediate step by converting our data to character first. We need to use *as.character* before *as.numeric*. This is because factors are stored internally as integers with a table to give the factor level labels. Just using *as.numeric* will only give the internal integer codes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "'data.frame':\t30000 obs. of  25 variables:\n",
      " $ ID             : num  1 2 3 4 5 6 7 8 9 10 ...\n",
      " $ LIMIT_BAL      : num  20000 120000 90000 50000 50000 50000 500000 100000 140000 20000 ...\n",
      " $ SEX            : num  2 2 2 2 1 1 1 2 2 1 ...\n",
      " $ EDUCATION      : num  2 2 2 2 2 1 1 2 3 3 ...\n",
      " $ MARRIAGE       : num  1 2 2 1 1 2 2 2 1 2 ...\n",
      " $ AGE            : num  24 26 34 37 57 37 29 23 28 35 ...\n",
      " $ PAY_0          : num  2 -1 0 0 -1 0 0 0 0 -2 ...\n",
      " $ PAY_2          : num  2 2 0 0 0 0 0 -1 0 -2 ...\n",
      " $ PAY_3          : num  -1 0 0 0 -1 0 0 -1 2 -2 ...\n",
      " $ PAY_4          : num  -1 0 0 0 0 0 0 0 0 -2 ...\n",
      " $ PAY_5          : num  -2 0 0 0 0 0 0 0 0 -1 ...\n",
      " $ PAY_6          : num  -2 2 0 0 0 0 0 -1 0 -1 ...\n",
      " $ BILL_AMT1      : num  3913 2682 29239 46990 8617 ...\n",
      " $ BILL_AMT2      : num  3102 1725 14027 48233 5670 ...\n",
      " $ BILL_AMT3      : num  689 2682 13559 49291 35835 ...\n",
      " $ BILL_AMT4      : num  0 3272 14331 28314 20940 ...\n",
      " $ BILL_AMT5      : num  0 3455 14948 28959 19146 ...\n",
      " $ BILL_AMT6      : num  0 3261 15549 29547 19131 ...\n",
      " $ PAY_AMT1       : num  0 0 1518 2000 2000 ...\n",
      " $ PAY_AMT2       : num  689 1000 1500 2019 36681 ...\n",
      " $ PAY_AMT3       : num  0 1000 1000 1200 10000 657 38000 0 432 0 ...\n",
      " $ PAY_AMT4       : num  0 1000 1000 1100 9000 ...\n",
      " $ PAY_AMT5       : num  0 0 1000 1069 689 ...\n",
      " $ PAY_AMT6       : num  0 2000 5000 1000 679 ...\n",
      " $ default_payment: num  1 1 0 0 0 0 0 0 0 0 ...\n"
     ]
    }
   ],
   "source": [
    "data[, 1:25] <- sapply(data[, 1:25], as.numeric)\n",
    "str(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When applied to a data frame, the summary() function is essentially applied to each column, and the results for all columns are shown together. For a continuous (numeric) variable like “age”, it returns the 5-number summary showing 5 descriptive statistic as these are numeric values. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "       ID          LIMIT_BAL            SEX          EDUCATION    \n",
       " Min.   :    1   Min.   :  10000   Min.   :1.000   Min.   :0.000  \n",
       " 1st Qu.: 7501   1st Qu.:  50000   1st Qu.:1.000   1st Qu.:1.000  \n",
       " Median :15000   Median : 140000   Median :2.000   Median :2.000  \n",
       " Mean   :15000   Mean   : 167484   Mean   :1.604   Mean   :1.853  \n",
       " 3rd Qu.:22500   3rd Qu.: 240000   3rd Qu.:2.000   3rd Qu.:2.000  \n",
       " Max.   :30000   Max.   :1000000   Max.   :2.000   Max.   :6.000  \n",
       "    MARRIAGE          AGE            PAY_0             PAY_2        \n",
       " Min.   :0.000   Min.   :21.00   Min.   :-2.0000   Min.   :-2.0000  \n",
       " 1st Qu.:1.000   1st Qu.:28.00   1st Qu.:-1.0000   1st Qu.:-1.0000  \n",
       " Median :2.000   Median :34.00   Median : 0.0000   Median : 0.0000  \n",
       " Mean   :1.552   Mean   :35.49   Mean   :-0.0167   Mean   :-0.1338  \n",
       " 3rd Qu.:2.000   3rd Qu.:41.00   3rd Qu.: 0.0000   3rd Qu.: 0.0000  \n",
       " Max.   :3.000   Max.   :79.00   Max.   : 8.0000   Max.   : 8.0000  \n",
       "     PAY_3             PAY_4             PAY_5             PAY_6        \n",
       " Min.   :-2.0000   Min.   :-2.0000   Min.   :-2.0000   Min.   :-2.0000  \n",
       " 1st Qu.:-1.0000   1st Qu.:-1.0000   1st Qu.:-1.0000   1st Qu.:-1.0000  \n",
       " Median : 0.0000   Median : 0.0000   Median : 0.0000   Median : 0.0000  \n",
       " Mean   :-0.1662   Mean   :-0.2207   Mean   :-0.2662   Mean   :-0.2911  \n",
       " 3rd Qu.: 0.0000   3rd Qu.: 0.0000   3rd Qu.: 0.0000   3rd Qu.: 0.0000  \n",
       " Max.   : 8.0000   Max.   : 8.0000   Max.   : 8.0000   Max.   : 8.0000  \n",
       "   BILL_AMT1         BILL_AMT2        BILL_AMT3         BILL_AMT4      \n",
       " Min.   :-165580   Min.   :-69777   Min.   :-157264   Min.   :-170000  \n",
       " 1st Qu.:   3559   1st Qu.:  2985   1st Qu.:   2666   1st Qu.:   2327  \n",
       " Median :  22382   Median : 21200   Median :  20089   Median :  19052  \n",
       " Mean   :  51223   Mean   : 49179   Mean   :  47013   Mean   :  43263  \n",
       " 3rd Qu.:  67091   3rd Qu.: 64006   3rd Qu.:  60165   3rd Qu.:  54506  \n",
       " Max.   : 964511   Max.   :983931   Max.   :1664089   Max.   : 891586  \n",
       "   BILL_AMT5        BILL_AMT6          PAY_AMT1         PAY_AMT2      \n",
       " Min.   :-81334   Min.   :-339603   Min.   :     0   Min.   :      0  \n",
       " 1st Qu.:  1763   1st Qu.:   1256   1st Qu.:  1000   1st Qu.:    833  \n",
       " Median : 18105   Median :  17071   Median :  2100   Median :   2009  \n",
       " Mean   : 40311   Mean   :  38872   Mean   :  5664   Mean   :   5921  \n",
       " 3rd Qu.: 50191   3rd Qu.:  49198   3rd Qu.:  5006   3rd Qu.:   5000  \n",
       " Max.   :927171   Max.   : 961664   Max.   :873552   Max.   :1684259  \n",
       "    PAY_AMT3         PAY_AMT4         PAY_AMT5           PAY_AMT6       \n",
       " Min.   :     0   Min.   :     0   Min.   :     0.0   Min.   :     0.0  \n",
       " 1st Qu.:   390   1st Qu.:   296   1st Qu.:   252.5   1st Qu.:   117.8  \n",
       " Median :  1800   Median :  1500   Median :  1500.0   Median :  1500.0  \n",
       " Mean   :  5226   Mean   :  4826   Mean   :  4799.4   Mean   :  5215.5  \n",
       " 3rd Qu.:  4505   3rd Qu.:  4013   3rd Qu.:  4031.5   3rd Qu.:  4000.0  \n",
       " Max.   :896040   Max.   :621000   Max.   :426529.0   Max.   :528666.0  \n",
       " default_payment \n",
       " Min.   :0.0000  \n",
       " 1st Qu.:0.0000  \n",
       " Median :0.0000  \n",
       " Mean   :0.2212  \n",
       " 3rd Qu.:0.0000  \n",
       " Max.   :1.0000  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "summary(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the introduce method, we can get to know the basc information about the dataframe, including the number of missing values in each variable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<thead><tr><th scope=col>rows</th><th scope=col>columns</th><th scope=col>discrete_columns</th><th scope=col>continuous_columns</th><th scope=col>all_missing_columns</th><th scope=col>total_missing_values</th><th scope=col>complete_rows</th><th scope=col>total_observations</th><th scope=col>memory_usage</th></tr></thead>\n",
       "<tbody>\n",
       "\t<tr><td>30000  </td><td>25     </td><td>0      </td><td>25     </td><td>0      </td><td>0      </td><td>30000  </td><td>750000 </td><td>6005824</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "\\begin{tabular}{r|lllllllll}\n",
       " rows & columns & discrete\\_columns & continuous\\_columns & all\\_missing\\_columns & total\\_missing\\_values & complete\\_rows & total\\_observations & memory\\_usage\\\\\n",
       "\\hline\n",
       "\t 30000   & 25      & 0       & 25      & 0       & 0       & 30000   & 750000  & 6005824\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "| rows | columns | discrete_columns | continuous_columns | all_missing_columns | total_missing_values | complete_rows | total_observations | memory_usage |\n",
       "|---|---|---|---|---|---|---|---|---|\n",
       "| 30000   | 25      | 0       | 25      | 0       | 0       | 30000   | 750000  | 6005824 |\n",
       "\n"
      ],
      "text/plain": [
       "  rows  columns discrete_columns continuous_columns all_missing_columns\n",
       "1 30000 25      0                25                 0                  \n",
       "  total_missing_values complete_rows total_observations memory_usage\n",
       "1 0                    30000         750000             6005824     "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "introduce(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can observe, there are no missing values in the dataframe.\n",
    "\n",
    "The information in summary above gives a sense of the continuous and categorical features in our dataset. However, evaluating these details against the [data description](https://code.datasciencedojo.com/datasciencedojo/datasets/tree/master/Default%20of%20Credit%20Card%20Clients) shows that categorical values such as **EDUCATION** and **MARRIAGE** have categories beyond those given in the data dictionary. We'll find out these extra categories using the value_counts method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<thead><tr><th scope=col>vars</th><th scope=col>n</th></tr></thead>\n",
       "<tbody>\n",
       "\t<tr><td>0    </td><td>   14</td></tr>\n",
       "\t<tr><td>1    </td><td>10585</td></tr>\n",
       "\t<tr><td>2    </td><td>14030</td></tr>\n",
       "\t<tr><td>3    </td><td> 4917</td></tr>\n",
       "\t<tr><td>4    </td><td>  123</td></tr>\n",
       "\t<tr><td>5    </td><td>  280</td></tr>\n",
       "\t<tr><td>6    </td><td>   51</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ll}\n",
       " vars & n\\\\\n",
       "\\hline\n",
       "\t 0     &    14\\\\\n",
       "\t 1     & 10585\\\\\n",
       "\t 2     & 14030\\\\\n",
       "\t 3     &  4917\\\\\n",
       "\t 4     &   123\\\\\n",
       "\t 5     &   280\\\\\n",
       "\t 6     &    51\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "| vars | n |\n",
       "|---|---|\n",
       "| 0     |    14 |\n",
       "| 1     | 10585 |\n",
       "| 2     | 14030 |\n",
       "| 3     |  4917 |\n",
       "| 4     |   123 |\n",
       "| 5     |   280 |\n",
       "| 6     |    51 |\n",
       "\n"
      ],
      "text/plain": [
       "  vars n    \n",
       "1 0       14\n",
       "2 1    10585\n",
       "3 2    14030\n",
       "4 3     4917\n",
       "5 4      123\n",
       "6 5      280\n",
       "7 6       51"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "count(data, vars = EDUCATION)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The data dictionary defines the following categories for EDUCATION: \"Education (1 =   graduate school; 2 = university; 3 = high school; 4 = others)\". However, we can also observe 0 along with numbers greater than 4, i.e. 5 and 6. Since we don't have any further details about it, we can assume 0 to be someone with no education experience and 0 along with 5 & 6 can be placed in others along with 4."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<thead><tr><th scope=col>vars</th><th scope=col>n</th></tr></thead>\n",
       "<tbody>\n",
       "\t<tr><td>0    </td><td>   54</td></tr>\n",
       "\t<tr><td>1    </td><td>13659</td></tr>\n",
       "\t<tr><td>2    </td><td>15964</td></tr>\n",
       "\t<tr><td>3    </td><td>  323</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ll}\n",
       " vars & n\\\\\n",
       "\\hline\n",
       "\t 0     &    54\\\\\n",
       "\t 1     & 13659\\\\\n",
       "\t 2     & 15964\\\\\n",
       "\t 3     &   323\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "| vars | n |\n",
       "|---|---|\n",
       "| 0     |    54 |\n",
       "| 1     | 13659 |\n",
       "| 2     | 15964 |\n",
       "| 3     |   323 |\n",
       "\n"
      ],
      "text/plain": [
       "  vars n    \n",
       "1 0       54\n",
       "2 1    13659\n",
       "3 2    15964\n",
       "4 3      323"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "count(data, vars = MARRIAGE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The data dictionary defines the following categories for MARRIAGE: \"Marital status (1 = married; 2 = single; 3 = others)\". Since the category 0 hasn't been defined anywhere in the data dictionary, we can incude it in the 'others' category marked as 3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "#replace 0's with NAN, replace others too\n",
    "data$EDUCATION[data$EDUCATION == 0] <- 4\n",
    "data$EDUCATION[data$EDUCATION == 5] <- 4\n",
    "data$EDUCATION[data$EDUCATION == 6] <- 4\n",
    "data$MARRIAGE[data$MARRIAGE == 0] <- 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<thead><tr><th scope=col>vars</th><th scope=col>n</th></tr></thead>\n",
       "<tbody>\n",
       "\t<tr><td>1    </td><td>13659</td></tr>\n",
       "\t<tr><td>2    </td><td>15964</td></tr>\n",
       "\t<tr><td>3    </td><td>  377</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ll}\n",
       " vars & n\\\\\n",
       "\\hline\n",
       "\t 1     & 13659\\\\\n",
       "\t 2     & 15964\\\\\n",
       "\t 3     &   377\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "| vars | n |\n",
       "|---|---|\n",
       "| 1     | 13659 |\n",
       "| 2     | 15964 |\n",
       "| 3     |   377 |\n",
       "\n"
      ],
      "text/plain": [
       "  vars n    \n",
       "1 1    13659\n",
       "2 2    15964\n",
       "3 3      377"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<thead><tr><th scope=col>vars</th><th scope=col>n</th></tr></thead>\n",
       "<tbody>\n",
       "\t<tr><td>1    </td><td>10585</td></tr>\n",
       "\t<tr><td>2    </td><td>14030</td></tr>\n",
       "\t<tr><td>3    </td><td> 4917</td></tr>\n",
       "\t<tr><td>4    </td><td>  468</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ll}\n",
       " vars & n\\\\\n",
       "\\hline\n",
       "\t 1     & 10585\\\\\n",
       "\t 2     & 14030\\\\\n",
       "\t 3     &  4917\\\\\n",
       "\t 4     &   468\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "| vars | n |\n",
       "|---|---|\n",
       "| 1     | 10585 |\n",
       "| 2     | 14030 |\n",
       "| 3     |  4917 |\n",
       "| 4     |   468 |\n",
       "\n"
      ],
      "text/plain": [
       "  vars n    \n",
       "1 1    10585\n",
       "2 2    14030\n",
       "3 3     4917\n",
       "4 4      468"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "count(data, vars = MARRIAGE)\n",
    "count(data, vars = EDUCATION)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll now move on to multi-variate analysis of our variables and draw a correlation heat map from DataExplorer library. The heatmap will enable us to find out the correlation between each variable. We are more interested in to find out the correlation between our predictor attributes with the target attribute default payment next month. The color scheme depicts the strength of correlation between 2 variables.\n",
    "\n",
    "This will be a simple way to quickly find out how much an impact a variable has on our final outcome. There are other ways as well to figure this out."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0gAAANICAIAAAByhViMAAAACXBIWXMAABJ0AAASdAHeZh94\nAAAgAElEQVR4nOzda4xk530f6Pd098zwJg5FtUnL61A2LZOWdaFhN7XadRIjJBJjvekSsNlo\nSQH6shCJEbAGvAYDJNmZDQINYGNDGsI6Wc7OOMgHAhquHBhwdyInwXISJ8iSMTs2R3bscCRL\nHtqywlHzMtSQM9PdVWc/1LBUqvvlf6pOnXqeD4Ou01W/evtW9Zv3nPecLM/zBADA4luZ9wAA\nAIih2AEAVIRiBwBQEYodAEBFKHYAABWh2AEAVIRiBwBQEYodAEBFrM17AExid3d3rPuvr6+P\n+5BFDywiU2AJA4vIFFjCwCIyBZYwsIjM6gWur68P+KwZOwCAishcUmwRNav9e+98X3jyn337\n7djAu997c2zgTfl+bODe6uHYwMNZ8N/Uwbl/HhuYUlp5z+3BiYePxOZla9H7E1ZXgwOjR1j/\n4EdiA1NKWZaFZ8YKH18BX3Fw4lvv7MUGppRWV4IHmcX/ZKKVfoB/ePOPhmf+8LdfTGbsAACW\nhGIHAFARih0AQEUodgAAFaHYAQBUhGIHAFARih0AQEXMutjVarVR7jPK3SYLBwCoqtLN2NVq\nta2tra2trSlDosYzF4s+fgBgLkpX7AAAmEz0RXt6ac0/dczDtc9LNT/V3NL8t31Lx8Obs3rt\nOR03O0L6jWpra6vn2HoOrHvwzYd3h4yY2W8MI44fAKBD4cWuvRK1l5ue5axVlQbcZ+gzdoSM\nPraeHW5oVPcXOFZm98c9x7+xsdF+c2dnJ6VUb7jOLwAsi8FXiW2axYxdS/vs1Ij3L24w/fL7\nbezXxjruP+6YR7x/s8m17O7uppTee+f7xnouAGBxNd/9B9e7mRa7DiOWvLmsJAh/UushAICi\nzbPYTbC3tOARDXrS9iPqJphKdLQcAFC0Uq+KnaxCLeKTAgBMb6bFbsCU29DZuH53mGYab+In\nnZLdsgBAEQrfFdtxOo+eH6eBSxbGymkZ5XQhPU93MniQE++HHfrFdnC6EwBgXFmeL+kpMyar\naCXZUVvcqtg/+/bbsYF3v/fm2MCb8v3YwL3Vw7GBh7Pgv6mDc/88NjCltPKe24MTDx+JzcvW\nov/buboaHBg9wvoHPxIbmFLKsiw8M1b4+Ar4ioMT33pnLzYwpbS6EjzILP4nE630A/zDm380\nPPOHv/1iGrYqttTH2AEAMLp5roqdgX5Hs002VzfZAwEAZqPixW5ADyvoTMIAAPNiVywAQEVU\nfMau2lYLOHT0nlf/MDjx7o/H5l29HpuXjoR/H+v12Lxrf/G/iw1MKR1ajf5PXfR3caX0R/2H\nO/TH0X99KX4pQd5oxAbGL+9YjX5fu/W22LzbL78RG5hSyg4HLwJL4Qsr4/+igwNfvPdnYwN/\n/OpXYwNTSlevvDn0PmbsAAAqQrEDAKgIxQ4AoCIUOwCAilDsAAAqQrEDAKiIipzupOcVJrpP\nKdxxpdd+F34d8YKw3XcbHNjvMhitobbfwfmQAYBxVaTYpZFrXPvGjps9H9VPd1q/h7fq2uBO\nOWLpBADoZ9l3xXZMpKlTAMDiWvZiFy6qKeqXAMC4qrMrdrBWweref9raMnoJG5A2/Thbo2rf\nvrGx0X5zZ2cnpQIu+QIAlNX6+vrQ+1Sn2HUvTRi9b4X3s8m0j6FjPDea3Lt2d3dTSuvve98s\nhwcAzNGNd/+B9a46xW5ALes3RTfZEw1Nm2AKsPXAyYYEAJCqVOwGG3CqkbmnAQCEWJZi1+8U\nJGVIAwAIUf1VsT33hw4+XfD0aSU5aA8AWCrLMmNXfh3tUCkEAMaV5U6ZsYCKWxWb//6LwYkf\n+3hs3rXr9djAI4eD561X6gexgVf2Y/NSSunQavRsfRact5JFJ5beoa//YXxo9LcxbzRiA7Pw\nH/Rq9ITFrbfF5uWX34gNTCllhw8HJ4Z3g/i/6ODAF+/92djAH7/61djAlNLVK2+mYatiq78r\nFgBgSdgV21u/I/DsIQUASkux602BAwAWjl2xAAAVYcZukRVydHlwZgFDDD6kN/7Y7fgFSUu3\njCCl1Cj9uq5GI3iEh1ZXYwNTSmkl+n/v+9FredYORQcGv68d/NFXYgPX7r0vNjCllF+/FhuY\nhf9csuBfxRd/+K/FBj749X8ZG/h2bNzIzNgBAFSEYgcAUBGKHQBARSh2AAAVodgBAFSEYgcA\nUBGKHQBARVTkPHY9rwDWffWIWq3WvrHj5tDtQ+82OLDfZcpaQ+24g6tfAABjqUixSyPXuPaN\nHTd7Pqqf7rR+D2/VtVE6pTIHAExs2XfFdkykjdjqAABKaNmLXbiJm6JOCQBMqTq7Ygdr1abu\n/aetLROUsLEeNUps+6jaP7WxsdF+c2dnJ+QZAYBFsb6+PvQ+1Sl23UsTRu9bsf1sYgMOwuto\ncru7u2m0HzAAUA2jvPtXp9gNqGX9pugme6KhaRNMASbLJgCAqVWn2A024FQjc08DAAixLMWu\n3ylIypDWCjFpBwBMo/qrYnsWpsGnC54+LXZvLwDAKJZlxq78Ju6aAABNWZ7n8x4DYytuVWz+\nlRdjA7OPPRgbePX6QWzgTUeC/3uT7e/FBl45yGIDU0qHVqNn6+PHWHaNRvCL581/9tXYwJRS\nWgn+Qef7+7GB2dqh2MC0FvwXffDHL8cGrt17X2xgSim/fi02MP7nEv2r+OIP/7XYwAe//i9j\nA99+/72xgSmlq1feTMPe/au/KxYAYEnYFdtbv72iDn0DAEpLsetNgQMAFo5dsQAAFWHGbpE1\nGvGZq6vxmaGyLPgo/fBD4Fej1xEUscCpER4aP8jgb2M9+u/ltuiDy8OPLk8ppZXgv+jspuh3\njegR1r8evNZh5bb3xAbmb38nNjCllKLXOuRX3ooN3HngU7GBD37jX8UGpjz6LXVOa1PN2AEA\nVIRiBwBQEYodAEBFKHYAABWh2AEAVIRiBwBQERU53UnPC0V0n2S4Vqu1b+y4OXT70LsNDux3\nNYvuoY44AACAdhUpdmnkGte+seNmz0f1053W7+GtPjdKp0z9r2YGADDYsu+K7ZhIm/tU2dwH\nAAAsrmUvduGmaYpaHQAwjWUpdq3O1H2sW2vL6L1qQFpBNr7X+vr6+vr6DJ4XACiJUd79q3OM\nXXfBGn32q+fRcjM2eAA7OzvtN3d3d1NK63feWfiwAIByuPHuP7DbVafYDWhFHZ1pyho3NK21\nZYIpQACAiVWn2A0Wu8O0iN2vHZmqHgAwrmUpdv1OQVKGtJ6BWh0AMK7qL57oWZImXvQwYloZ\nDtoDAJZN9YsdAMCSyPI8n/cYGFtxq2LzP3opNjD78E/GBl7bq8cGHloL/u/Nav0gNvA7+7F5\nKaW0tprFhwYLHmG90YgNvO1QbF5Kf/r16MSUVlaDA7Po35zoEda//nJsYHbzLcGBt94WG5hS\nSmvRv47Xrsbm7TzwqdjAB7/xr2IDUx78EvH29/9IbGBK6erbb6Zhq2LN2AEAVMSyLJ4YV78j\n8Bw2BwCUlmLXmwIHACwcu2IBACrCjN0i27sWnxm9mKYRHRi+2mcl/GDw6CNw8zx+oUOjEZ0Z\nnVevB38bb7+59MsIihD+B7Ma/K5R/+M/ig3MbolemrAW/CXnb1+JDUwp/rfxPz74mdjAjfNf\nig1M0cuhUqrIWlIzdgAAFaHYAQBUhGIHAFARih0AQEUodgAAFaHYAQBUhGIHAFARFTmPXc8r\ngHVfPaJWq7Vv7Lg5dPvQuw0O7HeZstZQ2+/g0hcAwLgqUuzSyDWufWPHzZ6P6qc7rd/DW3Vt\ncKccsXQCAPSz7LtiOybS1CkAYHEte7ELN3FTVCgBgCktS7FrFazuY91aW0YvYQPSCrLxvdbX\n19fX12fwvABASYzy7l+dY+y6C9boc2A9j5abi54H5KWUdnZ22m/u7u6mlNZvu2VmAwMA5uvG\nu//AbledYjeglnWUtilr3NC01pYJnqV1/5IUTQBggVSn2A0Wu8N0NrtfAQDGsizFrt8pSMqQ\n1goxRQcATKP6iyd6FqaJFz2MmFaeg/YAgOWxLDN25dfRDpVCAGBcWZ7n8x4DYytuVWz+tT8K\nDvzwT8YGXt9rxAbedHg1NjDbvx4b+NZ+FhuYUlpdiZ6tjx5jvR780nT7zcE/6JRFf81/ciE4\nMKWURf+g1w7F5tX/OPg1J7vlttjAtBY9A3JwEByY4n8b/+ODn4kN3Dj/pdjA7LbbYwNTCn7N\nefv9H4wNTCldffvNNGxVbPV3xQIALAm7YnvrdwSePaQAQGkpdr0pcADAwrErFgCgIszYLbLD\nN4VHZoePBAdGH9IbftB/Hn3AbBZ+uHr0CFMBX3UevKYl3X5r8EH6RXwTYxVwMHi8g6/sDL/T\nOFbv/oHYwBT/Ihb8F11/9ZuxgSml3/1Lj8cG/tSLz8QGpptujs3LD/ZjA7ND4a8581mcasYO\nAKAiFDsAgIpQ7AAAKkKxAwCoCMUOAKAiFDsAgIpQ7AAAKqIi57HreQWw7qtH1Gq19o0dN4du\nH3q3wYH9LlPWGmr7HVz6AgAYV0WKXRq5xrVv7LjZ81H9dKf1e3irrg3ulCOWTgCAfpZ9V2zH\nRJo6BQAsrmUvduEmbooKJQAwpWUpdq2C1X2sW2vL6CVsQFpBNr7X+vr6+vr6DJ4XACiJUd79\nq3OMXXfBGn0OrOfRcnPUPZidne+58Pbu7m5Kaf3OO2c6LABgfm68+w/sdtUpdgNqWUdPmrLG\nDU1rbSlVWQQAKq86xW6w2B2mhe5+VQcBgMksS7HrdwqSMqR1RGl1AMBkqr94omdVmnjRw4hp\nk+2H1eoAgGksy4xd+bVW5ra2KHkAwFiyPM/nPQbGVuCq2K/9YXDgfR+Jzbu+X48NPLQWPG+9\ncnAQG/jWXvwf6cpKFhsY/kLynlsOxQbGfxOjE7PdbwUnFuDgKzvD7zSO1bt/IDYwHT4Sm5dl\nwS8R9Ve/GRuYUvrdv/R4bOBPvfhMbGB2+x2xgeGyQ8GvOVe+/97YwJTStbcvp2GrYqu/KxYA\nYEnYFdtbvyPw7B4FAEpLsetNgQMAFo5ix/fI68HHhwUfyVXAsVxZ+Bijh9go4DjYRj049I7b\nDscGNqK/jeE/6Cz8l/vq1ejEVP/Tr8cGrhwNPrQ339+LDcwajdjAfC34jTL8eLiU0k/+u9PB\nieEvtQf7sXnZre+JDcyvBf8B1ot47R6BY+wAACpCsQMAqAjFDgCgIhQ7AICKUOwAACpCsQMA\nqAjFDgCgIipyHrueF4roPslwrVZr39hxc+j2oXcbHNjvahYdQx3x2QEAOlSk2KWRa1z7xo6b\nPR/VT3dav4e3+tyInXLoUwMA9LTsu2I7JtLmO1tmrg4AmMayF7tw0zRFrQ4AmMayFLtWweo+\n1q21ZfQSNiCtIBvfa319fX19fQbPCwCUxCjv/tU5xq67YI01VVby3aA7OzvtN3d3d1NK63cG\nX40bACitG+/+A7tddYrdgFrWUdqmrHFD01pbSl4WAYCKqU6xGyx2h6m1qwBACS1Lset3CpIy\npAEAhKj+4ome+0MnXvQwYpr9sADA7FW/2AEALIksz/N5j4GxFbcqNn/5K7GB2Yd+Ijbw2l49\nNvDIodXYwGx/LzbwzeC8Qtxx2+HYwEb0S1OWsuDA4LyUXvl6dGKq/2lwZnbk5tjAtBo8v5Ct\nRh9itBYcuPPh/yE2MKX0k//udGxgduSm4MBbb4sOfE9sYH7tamzgWz94f2xgSmn/6ltp2KpY\nM3YAABWxLIsnxtXvCDyHzQEApaXY9abAAQALx65YAICKMGO3yBqN+MjdS8GJ9eBD4A+tBf9v\n5KAe/G08dBC82OGOb12MDUwp5VffDg4MX4V1cBCbl0f/XBr7+7GBF3/lqdjAlFL9WvRXvRf8\nczl4K/iI9f03owO/GvzHsvGffiM2MKWUHToSG9i4cjk2MH/7Smxg/dVvxQaufehjsYG33xK8\npCyl9NoIv91m7AAAKkKxAwCoCMUOAKAiFDsAgIpQ7AAAKkKxAwCoCMUOAKAiqnAeu56X/2pd\nOqJWqzU/bn3Q/fDJrjPR/cDBT9HvMmWt0bbfwaUvAIBxVaHYpV41aOK6NqJWVxva7Vp1rX37\n0AcWPX4AoHrsigUAqAjFLljHLteJJ95M1wEA46rIrtgOs9kPm96tcYHP1XO/bUppY2Oj/ebO\nzk5K8RfTBABKa319feh9KlLsBqxLWCDtHbGjL95ocu/a3d1NKa3fcccshwcAzNGNd/+B9a4i\nxW6Wiyc6krsn7Vpbxh2D3a8AwDQqUuxmrBoThABAxSh2k+h3QhMAgDmyKnY8Pfeudp98OHxR\nBQDAUBWZseueM+t3BYiedxvx4YXqaIdKIQAwrizP83mPgbEVtyq2/vy54MT/9q8GB2bBeY1G\n8J/Aob13YgPTNy8GB6aUX307OjE6L/xsPgd7sXn5/n5s4MVfeSo2MKVUvxb8VTf2gn8uB29d\njQ3cfzM68KvBfywb/+k3YgNTStmhI7GBjSuXYwPDz8/VuPKd2MC1D30sNjC/+wdjA1NKr702\nfFWsXbEAABVRkV2x4fqth7CHFAAoLcWuNwUOAFg4ih3fK2/MewRDRB9iF35sWLxCDoQNj4wf\nZHBg3oj+3Y7+Y6lfDz5oL6XUuBac2Yge5MFb12IDww+JO/Sjt8YGFnE1yHztUHBi9CDzq8EH\nH6/9yP2xgentK8GBc3p7cYwdAEBFKHYAABWh2AEAVIRiBwBQEYodAEBFKHYAABWh2AEAVEQV\nzmPX8yoRrTMM12q15setD7ofPtnpiLsfOPgp+l3NojXajjs4STIAMJYqFLvUqwNNXNdG1Opq\nQ7tdq661b+83PGUOAJiYXbEAABWh2AXr2OU6+sRh0VOMAEDlVWRXbIfZ7IdN79a4qOdqb4Qd\nmRsbG+03d3Z2UirkgoMAQDmtr68PvU9Fit2AdQkLZMBBeDea3Lt2d3dTSut33DGzsQEA83Xj\n3X9gvatIsZvl4omO5O5Ju9aWscZgPywAMKWKFLsZq8YEIQBQMYrdJPqd0GQaFk8AAFOyKnY8\nPetX98mHp1xUoeQBABOoyIxd95xZvytA9LzbiA8v1OBLUwAADJXleT7vMTC24lbF1v+//zc4\n8ad/NjYvy2Lz0kEj+E/g8N47sYH5n/1JbGBKKV0NHmSKfiXJ68Fn88n3rscGpoP92Lyv//L/\nERuYUmpc3QsOvB78Ve+99nZw4MtXYgMP/eitsYEP/NN/EhuYUko33Rybl19+Izgw+jVn9Yc+\nGBuY9oN/t/N7748NTCm99tpradiqWLtiAQAqoiK7YsP12yvq0DcAoLQUu94UOABg4dgVCwBQ\nEWbsFtl+8GHRKaWUgtcm5Cn4mPqVLPh/I1n0CMOXEUQvF0kphX/NBSj9uq7s0OHYwPCFDiml\n+tXgJSP1t4MHGb7W4fD9t8UGrt1+U2xg+MKglFIWvTYofyf457L6Ix+KDUzhy6HCF2zN6TXM\njB0AQEUodgAAFaHYAQBUhGIHAFARih0AQEUodgAAFaHYAQBURBXOY9fz8l+tS0fUarXmx60P\nuh8+2XUmuh84+Cn6XaYsdV3oYuIhAQDLrArFLvW6AljR3ajV1YZ2slafa98+YHgD+h8AwAB2\nxZaLuToAYGKKXbCOXa5jFTWtDgCYRkV2xXaYzX7Y9G6Nm0Eb29jYaL+5s7OTUkpX3yn6eQGA\nklhfXx96n4oUuwoclza4IN5ocu/a3d1NKa3fekvhwwIAyuHGu//AeleRYjfLxRMdyd2Tdq0t\no4/BTlgAYHoVKXYzVsQEYUemqgcAjEuxm0S/E5oEBmp1AMC4rIodT8/K1X3y4ZktqgAAaKnI\njF33nFm/K0D0vNuIDwcAKLMsz/N5j4GxFbcqtv7iv4sNzH/6r8YGrq4ETzMf1BuxgYeuvx0b\nmL55MTgwpTz8XDnRryT5wX5w4P5ebGCWZbGBX/27/3tsYEqpfvV6cODbwd/Ga79/OTbw8P23\nxQau3X5TbOCH/u//MzYwpZQdOhwb2HjtUmzg6o98KDYw7QX/bqfo15zGBz8cG5hSev3119Kw\nVbF2xQIAVERFdsWG67cewi5aAKC0FLveFDgAYOEodotsrYAfX/QRbOHHcGYp+JC48h9kWswI\ny/91B8sOH5n3EIYIPx4upXRwJTjz+h+8FRt400ePxgauvSf4kLiVm4MPX0vRx/WmlOqv/3ls\n4Nr9H40NTI16bF74YbjxBwrHxo3MMXYAABWh2AEAVIRiBwBQEYodAEBFKHYAABWh2AEAVIRi\nBwBQEVU4j13Pq0S0zjBcq9WaH7c+6H74ZKcj7n7g4KfodzWL1mjb7+AMyQDAuKpQ7FKvGjRx\nXRtRq6sN7Xatuta+fegDix4/AFA9dsUCAFSEYhesY5fr6BNv5ucAgClVZFdsh9nsh03v1rgZ\ndLKNjY32mzs7OymltL9X9PMCACWxvr4+9D4VKXYD1iUslp4H5KVWk3vX7u5uSmn96O0zGxgA\nMF833v0H1ruKFLtZLp7oSO6etGttmWAM3Yt5AQBGVJFiN2OVmSAEAKpEsZtEvxOaTMMUHQAw\nJatix9OzfnWffHhmiyoAAFoqMmPXPWfW7woQPe824sML1dEOlUIAYFxZnufzHgNjK25VbP0/\n/HZsYOMTD8cGHlqNzUt79eDAw9ffjg3Mv3kxNjCllK4GDzJFv5DkB/vBiVkWHBjtP/8v/2t4\n5sGV67GB1//grdjAmz56NDZw7T03xQau3Hw4NvBHf/mXYgNTSo3Lr8cGrt3/0djA1Ah+qc2v\nXY0NTNF1qPGjH4kNTCm98fpradiqWLtiAQAqoiK7YsP1Ww9hDykAUFqKXW8KHACwcOyKBQCo\nCDN2iywroJdHH2AevzinEZyYpehj6uO/5EZ0Ysrr4ZnRP5e1Q7GB5bd3KXpFS0r7X38nNvDI\nR4LXbK3eGrw0YeVI8G/OyuHgN8r8evRR/ymtfWxj+J3G8vaV4MDoN5fsUPRLxMFBcOCcjNcM\nzp071zr47MKFC1mW1Wq17e3tAgYGAMB4xih2586de/jhh5s17tKlS0888URKaXt7W7cDACiD\nMYrdr//6r6eUXn755ZTS3/t7f297e/u55567ePFiSunMmTMFjQ8AgBGNcejAqVOnUkr33Xff\n+fPnT506dezYsYceeqj5KTN2AABzN8aM3ebmZkrp0qVLv/M7v5NS+jt/5++klC5cuND6FAAA\nczTGjN1jjz22vb199913p5Q2NzfvueeelNL999+fUvr0pz9d0PgAABjReDN2Z8+ebX7w+c9/\nvn3jI488UsjoAAAY2Xin53nkkUc6OlwZrtDQ8/JfrYHVarXmx60Puh8+2VfR/cDBT9HvMmWt\n0bbfoQzfWABgsVTkBMXdNWjiujaiVlcb2u1ada19+9AHFj1+AKB6xr50wblz55566qksy7Is\nSymdOHHilVdeKWBgAACMZ4xid/ny5c997nMPP/xw89TETSdPnvzABz7QXBtLSqljl+voE2/m\n5wCAKY1R7L70pS+dOnXq7NmzedvlP59//vmU0jPPPBM/tCnMZj9s6qpxxdn4Xuvr6+vr6zN4\nXgCgJEZ59x/jGLvHH388pdSxeOITn/hESunkyZOtdbJzMZt2NTPdxXRnZ6f95u7ubkpp/Y47\nZjosAGB+brz7D+x2Fk+MrSO5ewlFa4sFEADALI1R7E6fPv34448/++yz7ZN2zz77bPNT8UMr\nsUInCNVBAGAyYxS7T33qU9vb248++uijjz7a3NJcGLu5ufnJT36ykNGVVb8TmkxPqwMAJjbG\n4omjR49ubW1tbW0dO3asueXYsWNnz5595pln7rrrrmKGVzo9i1f3EorJ9sNqdQDANMY+xm5z\nc3Nzc/Ppp58uYjQT654z63cFiJ53G/HhhWqOwcUnAICJZe3nLhnsxIkT99xzz2OPPVbogBhF\ncati6//h3wQHfvyh2MDDK6P+xo5oP89iAw9dezs2MP+zb8QGppTyd4IHmVLwzyVbWY0NLL/z\nf+N/Ds/c//o7sYFHPnJ7bODabUdiA1dvDg5cuelQbOC9/9vfjQ1MKa3e9+HgxLevBAdmwa+0\nqVEPDjw4iM2r/8iPxwamlN54/bU0bFXsGLtiz58/3zzjCQAAJTTGrth/+A//4QMPPPDss88+\n9NBDlT+ort96CLtHAYDSGqPYfeADHxjw2dF36S4EBQ4AWDhj7IoFAKDMxpixq9icXCUU8BOJ\n/inn4YOMH2H0Ib1F/FxKLzscfMR6/KHWpX8FC1/okFI6dO8tsYELsNbh5sOxgatHghdPrP6F\ne2MDC5E3ogOD81IjfITRby5zes0xYwcAUBFjzNhlA/8DbT4PAGC+zNgBAFTEGMUu7/Lqq68e\nP358a2vLdB0AwNxNNWN31113PfHEE7VabXt7O2pAAABMZtpdsUePHk39T+cLAMDMjLF4oqcX\nXnghpbS5uRkxmAn1rJWtMwzXarXmx60Puh8+2emIux84+CkG1N/2R008HgBgycWsiv30pz8d\nMZjJdTehoutRq6sN7XatPjdKezP3CQBMbNpdsZubm2fPnn3kkUdCRrPkzNUBANNw5YlgHdN4\nQ3fCdjy2yKEBABU37TF25TSb/bCpq8YVZ2Njo/3mzs5OSikd7Bf9vABASayvrw+9z9jH2HXP\n2zWno+Y721T5Q9NuNLl37e7uppTW7zg6p+EAALN2491/YL0bXuw6TlM3+MJi8zLLxRMdyd2T\ndq0tjpkDAGZpeLH7hV/4haHnH37uueeCxrMYKj9BCAAsouHF7qGHHmrufu23K3YJ9TuhCQDA\nHI19rdjihrIQeu5d7V73aj8sADB7462KffbZZx999NGen5pv5+ueMxvx9L+ti1KM8nAAgDLL\nRi9kA1pdmnexWzbFrYqtv/BvYgMP/uuHYgOPZMG/aXtTn6a7w+FrV2ID8z/7k9jAlFL+ztux\ngdnhw7GBKXydVulfo3Y++j+GZx6695bYwMN33RobuHrzkdjAlZuDfxVXjxyKDeSnxkQAACAA\nSURBVLz3C/9XbGBKKd18c3DglbeCA8M1GsGB9Xps3sGP/HhsYErpzTdeT8NWxY7xlvbFL34x\npXTx4sXjx4+nlF599dU333yz+fHLL7881UgBAJjaGLtim2tj77nnno9//OMppW9961sPPPDA\nE088cfLkyWeeeebzn/98UWOch37rIeyiBQBKa5IrT9xzzz0ppddeey2ldPTo0ZTSyZMnK1bs\nFDgAYOGMsSv2ySefTCm98MIL73//+1NKv/7rv55SunDhQkEjAwBgLGMsnrhw4cL999+fUsrz\n/MSJEydPnmx96vTp04899lghA6SX5uKJO957Z3jyf/qTN2ID7/vBO2IDb77yWmxguu322Lx3\noi/BnD/zq7GBKaW9S5diAw++853YwL3XX48NvPatb8cGvr79tdjAjd//p7GBKaV0cBCbl9eD\nA9P+XmxeHh2YsuD1Vat/8a/GBqYClgZlKTox+jfnaj14fdXaavAPeu2P/yA2MKX02vt+IAUu\nnrjvvvteeumlY8eOpZQ+//nPnz59urn97NmzWh0AwNyN108feOCBp59+uvnxY4891jxl8SOP\nPFLAwAAAGE/wxCMAAPMydrE7d+7cU089lWVZ89KxJ06ceOWVVwoYGAAA4xmj2F2+fPlzn/vc\nww8//MQTT7Q2njx58gMf+IC1sQAAczdGsfvSl7506tSps2fPti+kff7551NKzzzzTPzQAAAY\nxxjF7vHHH08pdSyV+MQnPpFSaj/1CQAAcxF8wq2K6Xlhse6LUtRqtfaNHTeHbu/5dC59AQCM\na4xid/r06ccff/zZZ59tn7R79tlnm5+KH1o5jFjj2jd23Oz5qG4jtkMAgH7G2BX7qU99anNz\n89FHH22uh00pZVn26KOPbm5ufvKTnyxmeIuq2e1aN7U0AGAGxih2R48e3dra2traal58IqV0\n7Nixs2fPPvPMM3fddVcxw1teiiAAMK6xj7Hb3Nzc3NxsXX9iybWm4rp3v7a2jDVd15rn63jI\nxsZG+82dnZ2U0kG9Mc3gAYAFMvgqsU0WTwzRvX5i9JY2Qatr3bnjgc0m17K7u5tSuuO9d46Y\nDAAsuua7/+B6N7zYNY+oaz933VIZUMv6TdEV8VwAAEONPWO35D2vQ8/zoQAAzIVdsVPpPqfJ\nvEYCADDGqlja9dzr2nGWEwCAWTJjVxYdpdDxdgDAuBS7QQa0q36f6tg+Vj9T5gCAadgVCwBQ\nEaPO2LUuI9bzZrJOdjT9jsAzVwcATM+u2JlS4ACA4gwvdqbiAAAWghk7vke9HtzjG+H/Magf\nBAem4BGGf8WNa9eCE1Oqv/N2bODBleDAq3/+7djAN/7Z12ID79z8YGzgcoqfO1g7FJuXrQQf\njF7IbEnn4VFTawSP8sp+bF46tBb+NUc7CH+3GonFEwAAFaHYAQBUhGIHAFARih0AQEUodgAA\nFaHYAQBUhNOdDNLzQhHdJxmu1WrtGztuDt3e7+mczRgAGItiN8SINa59Y8fNno8a/ekAAEZk\nV2whmt2udXPEVgcAMA3FriyUPwBgSnbFTqXVxrp3v7a2jN7Y2if5Oh6ysbHRfnNnZyeldFBv\nTDN4AGCBrK+vD72PYjdE9/qJ0efVxmp1HckdD2w2uZbd3d2U0h3vvXPEZABg0TXf/QfXO8Vu\niAG1rN8UXfgTAQCMQrGbSs/zoQAAzIViN5Xuc5pMHGXxBAAwJatiJ9Szh3Wc5SQ2HABgMDN2\nZRFVCgGApaXYDTJg2qzfpzq2jzXxZpYOAJiGXbEAABVhxm6m+u1sNVcHAExPsZspBQ4AKI5d\nsQAAFWHGboGtrsb38oNGHhuY58GBqV4PDkxZqeNSauztRUem+tVrsYHXv/1abOAb/+xrsYHv\n/esfjA286f3fFxuYsvDfnQJE/0Vnhw7HBhbwbQwOzFP0q2JKK9GDfOtq8CvtobXlm0iKf7ca\nyfJ9owEAKkqxAwCoCMUOAKAiFDsAgIpQ7AAAKkKxAwCoCMUOAKAinMdukJ5XAOu+ekStVmvf\n2HFz6PZp7gkA0KLYDTFijWvf2HGz56MG6Hc9WQCAweyKLUSz27VumqsDAGZAsSsRrQ4AmIZd\nsVNpVbHu3a+tLSF1bWNjo/3mzs5OSgVcbhAAKKv19fWh91Hshug+4m30ljZWqxt8z2aTa9nd\n3U0pvW+EHzAAUA3Nd//B9U6xG2JA2eo3RTfBs9gJCwBMT7GbSuAK1o4oVQ8AGJdiN5Xuc5pE\n5Wh1AMC4rIqdUM/u1XGWEwCAWVLsAAAqQrEbZMD+0H6f6tg+2R5V+2EBgAkodgAAFWHxxEz1\nOwLPFB0AMD3FbqYUOACgOHbFAgBUhBm7BVavx18ttl5vxAbm4WOMTwwWPr7GtWvRkenaf7kU\nG/jt/+f3YwO/73/6aGzgke97X2zg2m23xgYW8budN+qxgdnaodjABZBl0XnBgSml77yzHxu4\nuho+yPBf7+ARNqL/APODg9jAEZmxAwCoCMUOAKAiFDsAgIpQ7AAAKkKxAwCoCMUOAKAiFDsA\ngIpwHrtBel4BrPvqEbVarX1jx82h23s+nWtUAADjUuyGGLHGtW/suNnzUd1GbIcAAP3YFVuI\nZrdr3dTSAIAZUOzKQvMDAKZkV+xUWlNx3btfW1tCpus2Njbab+7s7KSUDgq4ViwAUE7r6+tD\n76PYDdG9fmL0ljZBq2s9Xcejmk2uZXd3N6V0x3uDL2oOAJRW891/cL1T7IYYUMv6TdGFPJ3D\n8gCAcSl2U+l5PhQAgLlQ7KbSfU6TiaNM0QEAU7IqdkI9e1jHWU4AAGbJjF1ZdJRCs3cAwLgU\nu0EGtKt+n+rYPlY/U+YAgGnYFQsAUBFm7Gaq3xF45uoAgOkpdjOlwAEAxVHsFthKFp8ZfpWy\nPA9PbAQHrgQfkJDVY/PS9W9/OzgxpUtfPB8beNenH4gNvOn774oNXL35puDAW26NDcz39mID\nU0opRb9MxL/sFPBCFit6gG9eif9Br61GjzL8ipXhP+fyX1Nz7/pcntYxdgAAFaHYAQBUhGIH\nAFARih0AQEUodgAAFaHYAQBUhGIHAFARzmM3SM8LRXSfZLhWq7Vv7Lg5dHvPp3MqYwBgXIrd\nECPWuPaNHTd7PqrbiO0QAKAfu2IL0ex2rZtaGgAwA4pdWWh+AMCU7IqdSmsqrnv3a2tLyHTd\nxsZG+82dnZ2UUqNR/kvlAQAx1tfXh95HsRuie/3E6C1t4lbX/ahmk2vZ3d1NKd155/vGTQYA\nFlTz3X9wvVPshhhQy/pN0c1kXAAAnRS7qfQ8H8r0mdohADABxW4q3ec0mTJQqwMAJmZV7IR6\nNrCOs5yEZAIAjMiMXVk0G6GLTwAAE1PsBhlQrfp9qmP7WEtoRx8YAEA3u2IBACrCjN1M9TsC\nz3QdADA9xW6mFDgAoDh2xQIAVIQZuwV2UI+/Vuzh1eCuv7KSxQam1ehf2ix4hOFf8p9+4bnY\nwJTSX/iFh2MDj3zf98UGrtx0U3Dg4cPBgdEjzG65NTawECvBLxHRLxAp/GUxi36JKEIBlw4P\nDmzkwd/GLCv71dJXv/PWXJ7XjB0AQEUodgAAFaHYAQBUhGIHAFARih0AQEUodgAAFaHYAQBU\nhPPYDdLzCmDdV4+o1WrtGztuDt3e8+lcowIAGJdiN8SINa59Y8fNno/qNmI7BADox67YQjS7\nXeumlgYAzIBiBwBQEXbFTqU1Fde9+7W1JWS6bmNjo/3mzs5OSmlvvzFlLACwKNbX14feR7Eb\nonv9xOgtLarVpXebXMvu7m5K6fajd06fDAAshOa7/+B6p9gNMaCW9Zuim8m4AAA6KXZT6Xk+\nFACAuVDsptJ9TpN5jQQAwKrYCfXc69pxlhMAgFkyY1cWHaXQsXoAwLgUu0EGtKt+n+rYPlY/\nU+YAgGnYFQsAUBFm7Gaq3xF45uoAgOkpdjOlwAEAxVHsFtjhrB6e+fFbLscGvr5/e2xgfvmN\n2MDf+Op+bOA9f/mh2MCPXv9abGBKqdHIYwOD41LKogPjRxg9xGz/neDElOK/keE/mGhZ+A8m\n2h1FvPGGf9V59CUr438u0X/T4SN86L8PDkwpvTH8HdAxdgAAFaHYAQBUhGIHAFARih0AQEUo\ndgAAFaHYAQBUhGIHAFARzmMXrFar9TwLccc1J9rv43IUAEAIxS5Ss9V1d7uhW3Q4AGB6dsUW\nruccXrP/zWU8AEBVKXZzY5YOAIhlV2yY1sxcz72xU9rY2Gi/ubOzk1JKB8EXOQUASmt9fX3o\nfRS7WWvfA9sqfz13y7ZXwxtN7l27u7sppfU7jhYyRACgfG68+w+sd4pdjO7FEP0m7XqWObtl\nAYDpKXZhLIYAAOZLsQvTfUKTeY0EAFhOVsUGGHxCk55nNglfXQEAYMZuFrq73SjTe5ofADCW\nLM/zeY+BsRW4KvabF2PzXn/vX4gNfO+3LsQG/sal22ID7/nLD8UGfvT612IDU0qNRvAffvjr\nSBYdGD/C6CHesv9OcGJK8d/I8B9MuPAfTLi1w/GZ4V913ggOzML3EEb/TYd/D+O/5LT7xhtp\n2KpYu2IBACpCsQMAqAjFDgCgIhQ7AICKsCp2ge2l1fDMN2//r2ID77jlUGzgn995b2zgPT/2\nkdjAV/7tudjAB1eij2JOKWXRxx03ChhksLIv8Ni7KXgdT1qINSjRgeUf4dpq/JRKHv11Z9Ff\ndyN6pebKSvAIw9eSHtTnszjVjB0AQEUodgAAFaHYAQBUhGIHAFARih0AQEUodgAAFaHYAQBU\nxMKfx65Wq21tbY2yveeWnpmtu41yh6HP0jOq/T5DnwUAYBQLX+zSaI2teZ/uew4thSO2xsHP\nMnSLDgcATM+u2ML1bIHN/jeX8QAAVVWFYtdRkvpNp5XNQgwSAFggVdgVO1Sr6vXcT7oQz7Kx\nsdF+c2dnJ6W0d1D+q3MCADHW19eH3mcpit0APfeHDl3ZME1paw8cvEqj/VmaTa5ld3c3pXT7\nHXdOPAwAYLE03/0H17uKFLvWJNkoyxT6rY3odzDc4DuM8iw9A3s2PACAiVWk2A024jKFobtQ\nB9/BYggAYL6WotiNePq6hXgWAIB+qrAqtmnEE8ilgacaGXoWkp53GPwsoz8EAGAaSzFjN5Yp\nd8gOeEjHlvab4Us0AIAllOV5Pu8xMLbiVsW++Z3rsYF3vOdIbOC337gaG/jnd30kNvCVf3su\nNvBv/Dc/GBuYUkrhf/iN8p98J/y1LouN21s9HBuYwodY/m/iIoxwbTV+X1ke/XVn0V93I/o1\nZ2UleIThdeigHt+v3nrz9TRsVWx1dsUCACw5xQ4AoCIUOwCAilDsAAAqwqrYBXb4IHihQ0rp\nv7wevDThrtuDjwcPX+vwA5f+IDZw4/Wvxwam3VeDA1NK9YPgwCz6f4lZ9DHrq6vRgcGvn4eP\nFnCRwPBvY/gR69HzC+GHwIcfAH99vx4dGf9zDl8NFT7CApZ+Bg/x5rQfGzgiM3YAABWh2AEA\nVIRiBwBQEYodAEBFKHYAABWh2AEAVMQCnO6kVqv13L61tdXvs81PtR7efrPfxvac7vt3P6Tf\nqJoPHyt/xBECAAy2AMUu9WlaAz47VitqVq7BXbC5pX370O44OH+sBACAUVRzV2yzhI11/2ke\nPkF+9x1inxEAWELVLHaj6zdVFjV/NiBfkwMAYi3Grtj5apWzngfPRRkcvrGx0X5zZ2cnpZSu\nBV/+CwAorfX19aH3WYxiN3SFROXdaHLv2t3dTSmt33brnIYDAMzajXf/gfVuMYrdHDtcxyza\nHCftAAAGW4xiN1+zPBhOtwMAJrbsiyf6LWLoOO1cu6h87Q0AiFXNYtdxwrmOajX47MHtd+hZ\nvyZY0NqdP+DOFswCAJNZjF2xgxdPdH928Hnpep5VbuiVJyY2Qb5uBwBMIMvzfN5jYGzFrYr9\nyjeDT6HysR86Ghv44toHYwN/4NIfBAe+/vXYwOzoe2MDU0qpfhAcmEVP/2dZcODqanRg9H+M\nj94ZHJgK+DauBAfm0TuOwt/Uwt8j9w8a0ZHxP+dG9BjDR1hAewke4s1pPzYwpbT7nStp2KrY\nau6KBQBYQoodAEBFKHYAABWh2AEAVMRirIqlt7X4H9/H7rktNjB8rcODB1+LDdzLgw+YfePI\nfbGBhw/F/wes0Qg+7jiLPjQ6/FDr8BGuhAcWsJIti/71zqIHmWXBR+ln0YfAh7vpcPQ6ngI0\nopcmhP+9hAv/+7u2F504GjN2AAAVodgBAFSEYgcAUBGKHQBARSh2AAAVodgBAFSEYgcAUBHO\nYxesVqu1Pt7a2uq5vd3W1latVmu/Z/tDem4HAOhJsYvUUcU6bvZraT27nVYHAIzLrtgw3VWs\n2dhGeWzHPbU6AGACil2x9DMAYGbsio3UmnWboM+1dsj2nK7b2Nhov7mzs5NSSgf7Ew4UAFg0\n6+vrQ++T5dEX+iX1angDFk90PHCUUri7u5tSWr/j6FSj7Cn69+HFw/fHBj548LXYwL3oq6Rf\neSe4cB8+FD+z3mgE/6Cz6Ct8h18xPHyE4Rc1XylgD0qWyv9ziQ6M/pLDra6WfYQppUb0e0H4\n30u48DJ0fa8eHZmuvPVGGlbvzNgVor3PtT4e2tgGzNgBAAzlGLuyaPW50ZdcAAC0U+zCaGMA\nwHwpdpE6ut3oO1W7z3inJgIA43KMXZhmM+t35YnUZ0qv30F1DrYDAMZlVexCsio2kFWxIayK\nDQi0KjYk0KrYCFbFTm9eq2LtigUAqAjFDgCgIhQ7AICKUOwAACrCqthF1miER7545MdiAx/c\nezk28Ct/cjk28PvvvDk28K63vhkb+DvvxK+S2asH//KEHw6+uhr83861leAhhh8C/5Fv/W5s\nYEoFrE0IX28X/zoWPcIs+FcxX78rNjCllK0Gv5tne9djA/PwX8V6+NKE4N+cV+/+8djAlNKt\nq8PvY8YOAKAiFDsAgIpQ7AAAKkKxAwCoCMUOAKAiFDsAgIqo4OlOarXa1tbWgO0j3qFWq/V7\nip4Pb4WMcv/uMfQbFQDAiCpY7EbRs1d13Kf9DmO1rqE1rlUcNTkAIJBdsQAAFbGkxa5jT6vJ\nMwCgApZ0V+ws9dwPm94tlyMWyo2NjfabOzs7KaUUfckXAKC01tfXh95HsYs3YNXFxG40uXft\n7u6mlNZvf0/4EwEA5XTj3X9gvVveYteaMAvfDztg8UTHc1lCAQAEWt5iNy9FzOcBACTFbvaG\nnmYFAGAyS7oqtmnGe0J7PtfgMyEDAIyumjN2g882XNpnn++wAYBFl+V5Pu8xMLbiVsW+eOTH\nYgMf3Hs5NvArr1yJDfz+O2+ODbzrrW/GBv7OO0djA1NKe/VGbOBqFpuXVleD9yesrQQPcTX6\na/7It343NjCllLLoH0z4W0Yj+FcxpegRZsG/iivrd8UGppSy1eBpmjz8dFrhv4r1enBg9G/O\nK3f/eGxgSunW1Wtp2KrYpd4VCwBQJdXcFVu0fkfF2XMKAMyRYjcJBQ4AKCG7YgEAKsLiiYXU\nXDzxje97MDz5wev/OThxJfo/DwcHsXl7a0diAw+n4EN69/LV2MCU0lr0gf+N6BeS8KUJ9ej1\nIuHW8vCDwVNK4Ysnor+N0X/R6dDh4MBw4a+KRdi7Fhx4+KbgwPBvY3QdqhdQr954/bVk8QQA\nwJJQ7AAAKkKxAwCoCMUOAKAiFDsAgIpQ7AAAKkKxAwCoCFeemI9arda8fEXPq5O5sgUAMAHF\nbv66a1yr9gEAjM6u2DLa2trqOZMHADCAYgcAUBF2xS6GjY2N9ps7OzsppW/MaTAAwOwNvkps\nk2K3GJpNrmV3d3deIwEA5qL57j+43tkVCwBQEYodAEBFKHYAABWh2JWR89gBABOweGL+uk9Z\np9UBABNQ7OajVd10OAAgil2xAAAVodgBAFSEYgcAUBGKHQBARWR5ns97DIxt3EuKra+vx16F\nrPyBRWQKLGFgEZkCSxhYRKbAEgYWkVm9QJcUAwBYDjlL4Kd+6qeWLbCITIElDCwiU2AJA4vI\nFFjCwCIyly3QjB0AQEUodgAAFaHYAQBUhGIHAFARTncCAFARZuwAACpCsQMAqAjFDgCgLGq1\n2ogbe1LsAAAqYm3eAwAA4LvTcqPPz3WzKhYAoCxqtdrW1tbED1fsAAAqwq5YAIAS6bkrdsRp\nPMUOAKAsptwVa1UsAEBFKHYAABWh2AEAlMXW1pbTnQAAVEG/VjfigXeKHQBARVgVS8UNndCe\nZvERAJSKGTuW15RLygGgCK0pieab1FjvVhZPsIxqtZpWB0AJNd+e2t+hxlpOodixdLr/ZsZ9\n+Og3AWCWHGPHEmm2LhN10E/7/0w6/lImm+TuCByQP41pJuA7HhsywvaXmo7/7JXk9ccPusI/\naMWOopRqX6dKB0P1fOeb5q+mO3DAzdEzB2+PHfAEge0h3V9jGV4Y/aCr/YNW7Ki+MrySQrii\nV3xPcNT2DDRng2bzJjrleWKj+EG3VPsH3dI+ngmKrGJHxTX/Kgb80ZbqtQxG1/Odb0GfZSwl\nHFKh/KDnPZA5mOarVuyouOV8UWBJzPItPyRn+pBWVODxFcUdqtHxA5r4h+UHvSQ/6BDOY0dR\nqvo/rSKOJoGSGPDON82RUt071Po9ywT5UaWn+8xhadJBTnlJqBnwg07l/kH3TB4xVrGjKGWr\nOFFLllzKAkrF0qglsTw/6CnfPe2KZXKlOtp0gNj/Si7DywoLIfz/TuUP7Jk55VOU/6tewsCe\nmZX/QUdR7JhcOX+ne+oeatmWQQHA9Fx5goob8J8q3Q6AspnyvcmMHYWo/MEQrb5Y0AnWAVhm\n3d1uxLcYxY5IIecEL7+OWcCSLHFnCQ3+b33g6f7LE1hEpsASBhaRWf7AVqzFE8xfa4qu8uVm\n6L7dan/5lEr4L1v5A4vIFFjCwCIyyx8YQrFjKuWfohtQtpQwACrG4gkm1yxGTfMeyxDdE+ZR\nyybK/7UDsEAsnmBu2n/5ytxvulc5pHIPGICl1e8S5648wUw5xm6Uz87MgP/tTX8Uc/epnqc8\n6Ljjv6eB38BpfhzdF4trfTzlZUv8TwMojhk7YrS/V5Wk3BSk/AfthVfPnhVnmq+0iOvt9uuy\nIZPKId+BwdcIL8kvD7DoFDsitb9vpepOQvQ7AKKqX2+HVokv1dfbs3AXNEintgaKM+UbqGJH\nIcrzlj/0DXiyoZbnC5yXEu52L+GQChX+u13+wCIyBZYwsIjM8gd2PHDieqfYMZWO/VwlvAxD\nSYZRSVETV4E/o9aQQjKLm3ju6KCT9dHYL3YhAovIFFjCwCIyyx/YnZ8mOpjE4gkm1/HOlL73\nN68kcyezHEZ5dkAXtHgi8MjCnoGxnSxqAq/n/16mX4PSbspBhi9OL39gEZkCSxhYRGb5A6d8\nFsWOyRVxCHy4GQxjIc75soTK07NnpvzvWN74BZYns/yBPTOHv6nlMKnNzc3Rb85LocPY3Nxs\n5k/zLPP6NoYnlz+wiMxyBrZ+M6OUP7CITIElDCwis5yB/UKGJjvGjhJZiFPEJVN0lFX5ZyBM\n5AgsT2ZpAwfvbRgarthRIgMOipqm1Q0+wH+ClVD6HKVS2ven4gKLyBRYwsAiMssfOGWOYke5\ndK8zmv5IqdgetrUgF1JjGYQfR1j+wCIyBZYwsIjM8gd2JHcY8YkUO6bS8csXePKLwFWN4QJL\nJ0ys9ecWtdi2/IFFZAosYWARmeUPbE9ufx9s/2AUih2TK7TQlLbVtWs//0X5R0vFhP++lT+w\niEyBJQwsIrP8gYOfa/S3mJWiRwOTTeO1ZuzCxxNu6121Wm0hBgxAVZmxo3Ta9292H3I3rhn/\np2qyBxaxR5tlVv5jiRySJbA8maUNnGzPlRMUU7ixfi+LWBU74LmSI+SokPIv97NqUmB5Mssc\nWBt4YafBFLulM81am4mfccT82ZzHbsq/vaHTaXNviuEluPyBRWSWP7AjPIUemVr+wCIyBZYw\nsIjM8gdOw67Y5TLL+bAJDB5DyP+B0tR/e2X4Rg0W/uJS/sAiMssfmMo95VBQYBGZAksYWERm\n+QNDKHZLZMCbSkn+n9ExhsE3x4ptfjD3L3CA9mXtrY3TtM/YHc3lDywis+SB5Z9TNDUrsDyZ\n5Q/sCO/YMvpzKXZUXHEH5zV1hE/2dB2Paj+0YsqZxcAvv/yBRWSWOXCrbdl4yPDKH1hEpsAS\nBhaRWf7Alp6vD6O/aCh2VFz4396AecSQVtdu+pnU9i+/fWOFA4vILG1g++/elENalMAiMgWW\nMLCIzPIHhlDsmNyInWPuv+sz+Nsr1V91u/LvfbATJ0T7fvyo6cCSBxaRKbCEgUVklj9wGood\nS6Sgv73ww7BCcsK7ZvkDi8gsf2C78k9ImNcRWJ7M0gZ276sZ7w0rZ2lsbm5O/Nmoh4wVOPhm\nyNNN/1VPP6rwn8vEj1rowCIyyx84lyctf2ARmQJLGFhEZhkCN4cZ/HAzdsul+7CeZVbCPach\nyr/f0O5XgH6mfKFQ7JZIcceWBT5jR2BUE+1+Ty3Pu+yAFRKTDTL86yp/YBGZ5Q8EltDQ9wXF\njqkUcQB4uJ5/BgPq1Oz1XCOZCviGlP+IMQfeAUxDsaPiijiZyOBpxcnengt9Uw8/2VL5A4vI\nLH8ggGK3RIbu1izJu0vHmeGaH5RkbKlMIxlFayooanqy/IFFZJY/EKBJsVsiC/H+0e98v8v5\n/jfxHrryzy2ZogsJBOig2DG5WS5sLNXcRvvcZxFrMqZ8+y//ck6rYgGKMumZWSBeQeex6/fA\nkFPEtd8MOaddM2TKqO7THVU+sIjM8gcOfq5lCywis/yBLBvnsaNAhR60Fzu90fN0J1HhaYp9\nph2DCb98RdnOqF5cYBGZ5Q9kGZR/8tgcfKkodkyunL/T3ZrjDFm72tP0ra6472T7117OSyKG\nBxaRWf5AKiz8QJTyBxaRWf7Alp6xrY1Dn1Gxo0QG/J1M//dTdHma5uFFAPh2qAAAEMlJREFU\nH1Nf/pknE3ghgVRV61UicFK/5IFFZJY/MIRiR7ls9TpVb+WvhDazN/jw5PIHFpFZ/kCqpzXL\nG3vARpkDi8gseWDrzW6adz3FjtLp/vNYnre9qD10A9btVjWwiMzyB7Jsev7Xd5rfpfIHFpFZ\n2sCQmqjYMbmFOOPxgEGWYXg9TTmB1/GiMP1/JcsfWETmQgRO8/BFDCwis/yBHeElXwFgIcX0\npoxS7JhcOQ8v6DBgbBP/KXa8cBe3LKPM31jmrvx7hO1YDxT+Ylv+wCIyyxwYNVei2DGV9l2H\nadGKyGRrmhbra2RplXxOoojAIjJLEliSYcwysIjMkgdGRSl2BOjYdZi0H4AgJa8jRQQWkVn+\nwCiKHZGW/PQNy/lVA4Uq/85i+9xj9dsna1csc7CcM3bL+VUDc1fmI8YKCiwis2yB3Q8ca3ZQ\nsSPAgjabqGWJkx2rV6jw5R3lDywis/yBLKfwl9zyBxaRWf7AlrHeZRQ7plL+nY9FnO6k5EW2\n/Hsc7LiByYT/Z7L8gUVklj9wGoodkxt6juy5/34XMYYy/N1CWoQTsDmPXXhmaeeWTNGFBIZQ\n7JhcqX6VZ2ar+Eu7wijKP6doajZE+ZdzWhUbq+f/DRxjB99jyr+Tfg8s/55oYNGF/2ey/IFF\nZJY/sD15mocrdlRfv/9XTf//rfbzM9tFCxQk/D+T5Q8sIrP8gSGyPM/nPQYWVdT1Two1uG/F\ntrFS/W1TeeF/gOUPLCKz/IEDnqK0x4o55C4qqt2IsYodFTfLYgcwS2U7AdsMAovILFtg642p\n+4NR2BVLUXQmgEKVf+WHBTRRgc5jB1B9Hbt+2nffTHmS59IGLsQgiwucOGHhAovILH9gjByK\nsbm5Oe8h5PmwYUwwyM1hphgsjKH9l637d2+y3+2SBxaRuViBkyUsXGARmeUP7BnV+nj0/JV5\nF0tYMK3/lm31Md/hsZwWYl9SbGARmeUPZBl0nEilVqs5xg6+R/iJ4Dt2r3jtBiBQ+9vKuG8x\nih0VV1zr6jiDUaHPBUC1TbYGtptix+SKuCTiIirnOSoBWEKKHZNbiAbT8V+fwTcnformBwvx\nDQGghNoPrXOCYuiruGKnzzFf5b9kgitPCJwssIjM8ge2J0/ztqLYUXFFFDu7XAEoJ7tiYTyD\np8qTwgfAdLrPce10J1AUvQ2A4rQvj21uGeuSYk5QDABQEYodAEBF2BVL9XUcDOf0ewBUlVWx\nAAAl0j0BMfrh3YodAEBFOMYOAKAiHGMHADBnUZeysCsWAKBEprlIkl2xAABl0V3jmicoHvHh\nih0AQEUodgAAFeEYOwCAEnEeOwAA7IoFAKgKxQ4AoCIUOwCAilDsAAAqQrEDAKgIxQ4AoCIU\nO4AYWX/Th587d270awoBS8t57ABiDChw07/SNsO9YgODrc17AACVonsBc2RXLMDsXL58+cyZ\nM839s2fOnLl8+XL7Zy9cuPDUU081P1ur1Z599tnm9tZcYPsHHROE7VuaH7/yyiu1Wu3EiROj\nPPX29natVsuy7HOf+9y5c+eiv25gVnIAIozyorq5udn+Cnzs2LHWp1566aXul+izZ8/m3zsF\n2O+5uj97/PjxlNLp06eHPvXZs2c7nve5556b+vsBzIEZO4BIA1ZObG9vb29vN7tanudnz549\ndepUa3rs1KlTKaXnn3+++dmLFy+mlB599NHUtns3H2c/74c//OE8zx977LGhT918lldffTXP\n85dffjml9IUvfGHq7wQwB4odwIx8+ctfTik98sgjzZvND37v936vefPpp5/O8/zee+89f/78\n9vb2mTNnpny6hx56aMSnbk7m/eZv/ub58+fvu+++PM+3tramfHZgLqyKBYgxdOFqv2WzrYec\nOHHi5MmTPT/bEd79XO1b+n2231OfP3/+xIkT29vbKaVjx479/b//9++6667+XyhQXoodQIwp\ni92ZM2cef/zxY8eO/c2/+Tff9773vf/977/77rvTTIpd0yuvvPJLv/RLp06d2tzcfPLJJ++7\n775RvmqgVBQ7gBhDi93nPve5U6dO9btDx8MvX758xx13pNGK3aVLlwa0wKFP3e7cuXMPP/zw\n4C8EKC3H2AHMyM/8zM+klFonMXnhhReyLGudjqTpwoULKaXLly8/+eSTA6KaR8W98MILzTv/\n6q/+6jRP3TzRSfOpP/jBD7bygYVjxg4gxtAZu8uXL3/mM59pHsrWcvHixXvuuSel9OyzzzZX\np3Zon4Tb3NxsLmvouPPW1lbzgmP9ZuzGfeqtrS3dDhaRGTuAGTl69Oiv/dqvnT59unnz+PHj\nL7/8crNapZQeeeSRjk+1P/a5555rv/nII4+cPXu22b1GKWFDn7qV1uyOWh0sKDN2AAAVYcYO\nAKAiFDsAgIpQ7AAAKkKxAwCoCMUOAKAiFDsAgIpQ7AAAKkKxAwCoCMUOAKAiFDsAgIpQ7AAA\nKkKxAwCoCMUOAKAiFDsAgIpQ7AAAKkKxAwCoCMUOAKAiFDsAgIpQ7AAAKkKxAwCoCMUOAKAi\nFDsAgIpQ7AAAKkKxAwCoCMUOAKAiFDsAgIpQ7AAAKkKxAwCoCMUOAKAi1uY9AADGdvgXf7P1\n8d6vfPKOv7V9JMuOrGRHVrJbVrPmx2sr2dpKdmhlpfnBapYdXl1ZXclWV7LVLFtdyQ6trjQ/\nWFnJDq1mzU+tvPuplZUbW9ZWV1az7Nhf/7F//C8u3Lj57j0PrWatu628e8+VG4+6sXF1JVtd\nWVlt5a+srH334xv5K1me5XmW8ptuPnLwzjtZylfyPEt5OthP9Xqq11PjIB0cpPrBjZv1ev7u\np/J6vXm3vHHjZvNTeb2eGvX8YD9/9255o57vH6RG/dbHfvE7/+iX84OD736qXm/e7PFxvdHY\n38/rjebNxv5BflDP641G89+95s16Y7+Rv33QuF7P9xr59Xp+vf6Tly+8mP1w82f0YP6NOf2m\nsHTM2AEAVIRiBwBQEYodAEBFKHYAABWh2AEstvaFFMU58+WXZ/As+1evzeBZrpz6BzN4lp21\ne2fwLNBBsQMAqAjFDgCgIhQ7AICKUOwAACpCsQMAqAjFDgCgIhQ7AICKUOwAACpCsQMAqAjF\nDgCgIhQ7AICKUOwAACpCsQMAqAjFDgCgIhQ7AICKUOwAACpCsQO+69y5cydOnMiyLMuyEydO\nvPDCC7N89ubzjnLPc+fO1Wq1CR44llqt9tRTT507d+7SpUsdn7p06dK5c+eeeuqp9mEM0DFg\ngIJkeZ7PewzA/F26dOmzn/3s9vZ2x/bjx49//vOfn80YmuVslBeljnuO/sAJxpNSOn369GOP\nPdb+qTNnzjz++OPNjycY8PQO/+Jvtj7e+5VP3vG3to9k2ZGV7MhKdstq1vx4bSVbW8kOraw0\nP1jNssOrK6sr2epKtpplqyvZodWV5gcrK9mh1az5qZV3P7WycmPL2urKapYd++s/9o//xYUb\nN9+956HVrHW3lXfvuXLjUTc2rq5kqysrq638lZW17358I38ly7M8z1J+081HDt55J0v5Sp5n\nKU8H+6leT/V6ahykg4NUP7hxs17P3/1UXq8375Y3btxsfiqv11Ojnh/s5+/eLW/U8/2D1Kjf\n+tgvfucf/XJ+cPDdT9XrzZs9Pq43Gvv7eb3RvNnYP8gP6nm90Wj+u9e8WW/sN/K3DxrX6/le\nI79ez6/Xf/LyhRezH27+jB7MvxH1o4fBzNgBKaXUbHWnT59+9dVX8zzP8/yll17a3Nw8efLk\nuXPn5j26IZoDLij82LFj3X13e3v72LFjBT0jwMQUOyC98MIL29vbTz755GOPPXbXXXc1Nz7w\nwAPNubovfOELrXteunTpzJkzzV2fZ86cad9H2dz4yiuv1Gq1EydO9NySUrp8+XJ7wuXLl/uN\n6sKFC0899VTznrVa7dlnn209UfcH7btihw7y0qVLzeT22H5+7ud+bnt7+8KFC+0D297e/rmf\n+7mOe/b70roHPPj70PP7BjAKxQ5I//7f//uU0ubmZsf2Bx544OLFi1tbW82bly9f/uxnP9va\nBfn4449/9rOf7WhmZ86c2d7evueee/pt+cxnPtOe8Lf/9t/uOaTz58/ff//9TzzxRPPm9vb2\no48+OrSEjTjIz372s83kUWLvv//+lNLLL7/c2tL8uLm93Yhf2oh37v5OAgyl2AGp2XLuu+++\n7k+1F4vf+q3f2t7ePn78eHPX5/Hjx7e3t3/rt36r/f4f/vCH8zxvPyKtfcv29vb29vbZs2eb\nCWfPnj116lTPXb2nTp1KKT3//PPNe168eDGl9Oijj6a2I9V67n4dZZAPPPDAm2++mef5c889\nl1L64he/OOCbc999921ubn75y19ubWl+3PHtGvCldQ94lO9D93cSYCjFDhjVb//2b6eUfv7n\nf755s/lBc2PLQw891PGo9i3NSvTII480bzY/+L3f+73u53r66afzPL/33nvPnz+/vb195syZ\nwEH+/M///NGjR1tj6z6ErsPm5uapU6eau3QvXbp06tSp06dPd9xn9C9txDt3fyf7aV9IUZwz\nX355+J2mtn/12gye5cqpfzCDZ9lZu3cGzwIdrIoFRl2z2X239i2DP9u+pVvPhBMnTpw8eXKU\ne44+jFEG2f2p8+fP/8RP/MRzzz330EMPnTt37uGHH37++ec/8YlPdCeP+KWNdWeA0ZmxA9KT\nTz6ZUmpfH9Cu3/ZCnTlz5uTJk8eOHXvuuedeeumlV199dfZjaPmhH/qhlNK//tf/uvXvhz70\noTmOB6AfxQ5IP/3TP5167ZG8cOFCrVZr7cdsnuCjtci0+cFYZ/1o3jnv0n3P5sKCp59++qGH\nHnrggQeOHDky1lNMM8huR48ePX78+MmTJy9fvnzy5Mnjx4839+R2P+8oX9q4dwYYnWIHpE98\n4hObm5tPPPFE+8lBzp8//8QTT2xvb//sz/5sc8vP/MzPpJR+9Vd/tXmz+UFz44iad26tQn3h\nhReal7jod//mZOHly5ebc4qjP8U0g+zpr/yVv5JS+rVf+7WU0sc//vF+zzvilzbu9wFgVN3/\nZQSW0KuvvtpzWqu1cjPP8zfffLPjlCibm5vN5aV528LPlu4t3QkppYsXL3bf/+zZswNeslrP\n3v3A6QfZ81Pt+4J7DniUL6014NG/DwBj8doBfNdLL710+vTpZuc4fvx462wjLa+++mprQWj7\nZSrykTtTe8Lx48dffvnlfvfvuFv7Z///9u5oBUEYAMMoQe//zF0EMmaZ02nbzzmXBZuzLj6a\n5vtvSj6GXZeD/PjW+7Qsk27PWy2tOuCm8wCwn7tiAQBCuMYOACCEsAMACCHsAABCCDsAgBDP\nfx8AAM0ej61b38pHlh27Q27PCNWD0Zomah1/zFVU41z6icBOwg5gMt8eNbu8W6bDdnCcHOFw\nb/0cf/xVVHN1ORI4z1YswExkwWh8IgxF2AHMZJCGyKiZLqsIOA8ksRULwBEZ141lrAIWwg6A\nIzKuG8tYBSxsxQLQLCOAMlYBJb/YAQzqnl1Ce5GQRNgBDOqezOrylyWTylgFlGzFAnBKRh5l\nrAJ8jwHms66Q8pXrntnQa5arx79zlvWY180CPwk7AIAQtmIBAEIIOwCAEMIOACCEsAMACCHs\nAABCCDsAgBDCDgAghLADAAgh7AAAQgg7AIAQwg4AIISwAwAIIewAAEIIOwCAEMIOACCEsAMA\nCCHsAABCCDsAgBDCDgAghLADAAgh7AAAQrwACPmHeyFXR24AAAAASUVORK5CYII=",
      "text/plain": [
       "plot without title"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_correlation(na.omit(data), maxcat = 5L)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can observe the week correlation of AGE, BILL_AMT1, BILL_AMT2, BILL_AMT3, BILL_AMT4, BILL_AMT5, BILL_AMT6 with our target variable.\n",
    "\n",
    "Now let's have a univariate analysis of our variables. We'll start with the categorical variables and have a quick check on the frequency of distribution of categories. The code below will allow us to observe the required graphs. We'll first draw distribution for all PAY variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0gAAANICAMAAADKOT/pAAAAPFBMVEUAAAAaGhozMzNNTU1Z\nWVloaGh8fHyMjIyampqnp6eysrK9vb3Hx8fQ0NDZ2dnh4eHp6enr6+vw8PD////GSW4mAAAA\nCXBIWXMAABJ0AAASdAHeZh94AAAgAElEQVR4nO2di5qiuBZGdai+zZnpadv3f9fTWiAh5LJz\ng524/m+mkBAhi7AMIq2XOyGkOJezG0DICEEkQioEkQipEEQipEIQiZAKQSRCKgSRCKkQRCKk\nQqqL9FNZxkGBRDMJInWDAolmEkTqBgUSzSSI1A0KJJpJEKkbFEg0kyBSNyiQaCZBpG5QINFM\ngkjdoECimQSRukGBRDMJInWDAolmEkTqBgUSzSS6RLou00deD+bHpagtUNbmXY3W+8DWClcT\nTEjXuFMqoEh7akQSVSJdN2xXN35i2qJcX5Ngn81gc+dcrR5UIlIxivhVYUQSjSIZ7G8g0lUK\n2bhTEKmIRLNIQphw2qIk9tk8ym5HpD5FcqP0KFIdEk0iXY3/E2DCaYuyOQn19tkKpl6kQhRF\nIh1Molak9U1f/xcb+hCpBoqMpQOSni82vEgM9j5GpMhZxHVT5/rTPvqUiFQDRYdIVUj6HZHW\na48ryRgiGWC9i1QFpQeSnkUy8O2H+WmLwoi0qTYKyVUKszZMj0g7fa5CGClqCxRJn63FC1G3\nIkVQpL2lnmQUkXq9s2Gesdrci0jFKOKuUk/S+50NLTIOCiSaSRCpGxRINJMgUjOU8tPSbU7s\nlMooI5IgUjcokGgmQaRuUCDRTIJI3aBAopkEkbpBgUQzCSJ1gwKJZhJE6gYFEs0k1UW6PfL5\n1467NKly+ipKUZxbEM9lPs05V0Ri7aFzZ0v7ZNfdGQWVnvMKIkVQnFtApKLZ0j7pXKTpT0LT\nEGmgFJGEc0UkiNTmOa/IRZrmP75pkDRQikjCuSISRGrznFcQKdJpzi0gUtFsaZ/0LdIziJQ2\nh0iu2dI+GV6kv/7E9ayPRzL22snx7cIHjX/3diCSDdC5SDsc7SJ9XlTIGJE+PlFHGZEQqXC2\ntE96F+mee2qHSDXmikgQKVqCSIgUJxlFJFdUvHE45KodItWYKyIZRSTXSjsbkRDJxPFvHZGi\ns6V90rdIBXc2IFKNuSISRIqWHCZSGSkiIRIiVSBFJERCpAqkiIRIiFSBFJEQCZEKSD+WIBIi\nIVI+KSLVmqvWRTo+wcyLY+8gkre9qaWIFCVhRIqWIFIrkXzp7wUdkRBJUsqIFCVBpGhJLyI5\ncH3tTS1FpCiJZpE+74uZ5jtk8m6cubm2EypApMTKpSiOLSBS4ey2cbMnsy2Zt3LeXNsJFSBS\nYuVSFMcWEKlwdtO26Y5I7j7bbhCRas0VkegVyXIFkTzZiNR6Y3Xj24WIVDhrte71VijyTQah\nrwQp2Dl1wogUQXFsAZEKZ63WTeYfRiQPKSLVmisiUS/S8giRPKSIVGuuiASRoiWIhEhxEvUi\ncWq367PtBhGp1lwRSRciFXxt4s21nVABIiVWLkVxbAGRCmet1q13NoSmRp8g0q5l7vamliJS\nlESzSMl98m4ifbgTb29qKSJFSRApWtKdSOHjMKf0LUVKOyFCpGgJIr2lSFPatS5EipYgUoJI\nKS/jqkWaEi8aI1K0BJHkIiW9jKsWKfXTF0SKliCSWKS0l/ExRPrLcavnA6C0n0+KY++8tUim\nTYef2r2VSDMJI1K0BJEaiOR6HX9Fxws6InlxlIuU/NFzxyKtKI4tMCIVzsqPOVFUvLC1/H0k\nREIkRqRdEGkJIhXOio+5fVwr7UykpfcQSZFI3NngwuleJOsdekykgv2Zm4FESgkiKRNpuo8w\nIo1wZ0NaEAmRJKVykdI7zbEFRCqcLe2T7kUKnhjtSREJkRDJkWn9W0WkeQcgkmyuiASRoiWH\niTQZE0Tybh2RorOlfdK3SFPyzwUgEiIhUgVSREIkRKpAikiIhEgVSBEJkSQipd2jsV8pIiFS\nylwRiWKREr99Yr9SREKklLkiEr0ipX77xH6liGTYFG9zoBSRoiR6RUq9j32/UkRCpJS5IpKd\nSJsd34dIb/tDY4ikRSQrrx1fb5UlYURyxdggImkRaYwR6dUniNSXSL7oeS2XBpG6FUkskWKR\nfFsIH4aMSNFZq3WI5O6zGyJ5GoZIiJTQZzdE8jQMkfwicWeDO0kiNWpDaXy7EJEKZ0v75C1E\nmjeYJJJrVGJEsueKSBApWoJIiBQnQaRoCSIhUpwEkaIliIRIcRJEipYgEiLFSRApWoJIiBQn\nQaRoySAi2TYhkj1XRIJI0RJEQqQ4CSJFSxAJkeIko4jkyoOk8irTg0gRFMcWEKlwtrRPGJEQ\nqWCuiGR0kWJHDyLF2GKVS1EcW9AkkvTrbxEpWoJIySLJjr4+RDImoX98gEjREkRKF8mY+I8+\nRGo1m390DSJS+B+ObEgRKbaoikiTOUUk32aViRT+SswtqWaRhEdfFyItJ6lelL8cX2L12t/C\njlcUV3f3JlLkKzG3pHkiGbujpUjRo895+L2i6DAU/wNtRqRoydGndgOI5CHocERaeBBpKJGM\nl/F8kQr2ciILIrk3h0jy57zyjiPS3Wp5xyJxandDpNNEGujUbjL+RyTvZhEpzhar7MEQHH09\niCT+Eiv9IiX/RPEOx7/ZtxRp3ictT+1kR18XIsnShUjbYyx8eG2CSGeJJIxvC4hUOOts49uJ\n1PLOBkQKzBWRqBdpMqejiyTrsxsieRqGSAGRZJ+Su5660BTsoPIgUgTFsQVEKpx1NVF8IX+/\nUptmX4MRKc4WKEaklUS9SM8gktlntwoiWfslwBYoRqSVBJGiJYiESHES9SJxarfrs9u7iLS0\nsl+Rlt2sRCTRPRr7lSISIqXMFZGoF6nwh8YQCZGkc0Uk+kVK6BNEKrYJkTJJEClagkiIFCdB\npGgJIiFSnASRoiWIhEiJ2QDUW+0RcXU3IikXyZf+jkNGpP5EmlNLpEbNE8W3C+3jUP+IhEj9\niTRvsJZIUbZAMad2KwkiRUvGFim2g0LFiLSSIFK0BJEQKU4yikiubHAqr1scRIqgOLYQamB/\nIj2PPseOazhb2ieeEcnoC0YkRPLOFZEgUrQEkRApToJI0RJEQqQ4CSJFS8YXKbSDQsWItJIg\nUrQEkRApThIWyaJAJPFzXkGkCIpjC+HDEJGis6V9gkgF8bEFihFpJUGkaMmbiPS5kxApk+Rt\nRPIcJ4gU2UE+5FdpKYpjC6EGdizSAoBICc95BZEiKI4thI9DRIrOlvYJItXIjs6J/CotRXFs\nIdw2RIrOlvbJoCK1+DWKQHZ0TuRXaT5JhkiP6BBJ9kNjnh2sSiT513GFDpMORGry+0iRxJFf\npfkkmSJt2neWSMKfvpTt4VNFSviCyNAhgkii/tYm0qtt3Ypk7Nw+RdoQIJI4O+bb6SLF2qle\nJE/rOxJpbXdfIvl+CaqTWCRjoIxDovvwajUiueIuTaqcvop8khpXDXSe2p09W0SyX2mt0SXn\nOa8gkpcEkVrNFpHsV4pIoWJEsueKOgWR2jznFUTykiBSq9kikv1KESlUjEj2XFGnIFKb57xS\n/c6GZ5IurqRUbrbiZxwk6WsOLMt8WjpIFCW8xrOWOiPplPhqBRsuq9LmK4s7FanKmnWIFElH\nIkmCSKmVEalOEKl2FUTKCSI1WZqdYUUi5M2CSIRUCCIRUiGIREiFIBIhFYJIhFRIbZGWz6Gl\nNwmIKyetOLUVabHXPs0PNvNrlnnHorWOZ7WSaT5AGGFp7Xb6WjqFli7rMpe3ILFw/Ci7abDa\nq9XyDqos0nJnlH3bmrf+dJdVTlpxaivSslv7tC+fpvmhPZ2rTNuqc51pU0c4zQcII3w26LOB\nkzG3qTWZtWz8DVQTEgvnbjzybmxaYT7b5qg2uTsx1PxzRZrWXnwvkayqs1zdiTRtar2WTtul\nukQyGu0RaVIg0sIg2zdLwwWVJ3OqQ6Rlxm6aU6B9X0y2SMLDraZIAQTjMAuLtNZyLT1cJDdK\nkkhWm0cTaRKfBh4t0rRt2lLk7wvrJH192mki7REMCK9I07aW3b5zRHKiZIm068RQ8xuIlHS8\ni0VaDzeJoQnWJcdcq6fLIiPSS6DXEXiiSP6jzlbBI5Jv6UkiRTbS0YgkdiOnstjQCj0UWftm\n5e4dHBJpORQ1iBRGCIt09y09R6QASqJI+04Mrbm+SNP2T6DiOnYKKs9/dIk0mYVikYxuOl8k\nAULgqh0iNRPJOMIk+2YSVlYp0pY1KNBeJB2ndl6EuylQSCRO7Z7T6h/IuuAi9XsVyWA1MeZ9\n77/YsLm4eq5IfoT7ej0+LtJ0VyXSHiVXpNMuNkzrtahJtGum5VmSNUvrplVOjvXJ/W5ryzmr\nPV2qTK8q8g/OA9NsgAjCutic2jvAvdTcP3LCzGyfLtxYkHztoV3n+abca0dIhSASIRWCSIRU\nCCIRUiGIREiFIBIhFYJIhFQIIhFSIYhESIUgEiEVgkiEVAgiEVIhiERIhVQX6aeyjIMCiWYS\nROoGBRLNJIjUDQokmkkQqRsUSDSTIFI3KJBoJkGkblAg0UyCSN2gQKKZBJG6QYFEMwkidYMC\niWYSROoGBRLNJIjUDQokmklOEen6mcej53+fhY56mwXX5XnXq+8JQdQWKBVIVqCzSKy2fLbm\numn42sSr+YyrvXzzrONJJNliFmZt2DkirUihw++69NpPc/JaqkKkchJxfzYjudqPrtZDw6yr\nRWksXx+qF2lDW5K1Yd2KdBXuh7YoY4pkHWxmQ50imVXN4exwEknCpw6JWRvWg0hmF22fm4Ta\nAqWcRN6ZzUgiIl23dXcirecIiFQxQpafrxczL9X19b92kUpI5GfrzUgSRLIHqL1IZpXDSSQZ\nSiTRW/Tg4SfeDW1R6pDIUJqRGBcb1gYjkiRrw/Se2l03da77A0+FSKUk7mccSyIfkXavAA6R\nBCcLbfskkvcTyXiZ3Ikkv37ZFqWYxP2MY0kSRNp9+uAQ6Wf0OlDbPonk/UTa1HEcfmOMSBpO\n7XaPNiLtnLL/2lURqUaELCuSh2otXjqoV5HCJNf9E44m2e3Sq/VnebgBNJq/1poL+xCp3KPT\nRdq8RV8+I9+ernUiUjmJgqt2rjsbNg1bGdeWb14NuLOBe+26QYFEMwkidYMCiWYSdSJd6427\nz5yHMg5J7YxIok6k2hkHBRLNJIjUDQokmkkQqRsUSDSTIFI3KJBoJhGJNH3+fWSe3h1TlaRD\ndhokWrI2TCLS7MlsyzT/sac6Sa1OS3lJ0IYiPdYcObvpVkYkEYg03VNEut1vm6TMFjzVO7tF\nSSHZocTmyxZHqxfEs8qzCstJtLCsDZOf2k3WTIcipb0kIFKrwnISLSxrwxJEWs6H5hLH4ffX\nnxTspAOS8pKASK0Ky0m0sKwNSxyRHIddRyNS2kuC8teElLj30WmF5SRaWNaGJVy1mx8NIJKM\nhBGpVWE5iRaWtWHvKJKQBJFaFZaTaGFZG3bsqd3HnwRaqVukXeMHFeljD5rydEGhp5XrZxOu\n6YbksbaP01IuUuSdxZNUv0iZp3aIVKvQc5QFO2dL0rNI3pcK6yWjD5EELwmIFKuZW+g5yMYX\nKanP9IskfElApFjN3EJPKxFp02eqRSpBQaRahZ5WxkSyPpI4TyRX4xFJjIJItQo9rWRE2vQZ\nIuUtRiRE2vQZIuUtRiRE2vQZIuUtRiRE2vQZIuUtRiRE2vQZIuUtRqTx72xI6jNEyluMSAkk\niIRI2dVLSNyrdBciEiI5ZuuhdCxSUp7HymFbS4yxexDJe7QgUmhxCYl7le7Cjz1oytMFheUk\niIRI2dVLSNyrdBciEiI5ZuuhIFKtwnKSNxApmEcrDt1gzXTceETqTyRGpMzFjEiJOU8kV2sQ\nSYyCSLUKy0kYkRApu3oJiXuV7kJEQiTHbD0URKpVWE6CSIiUXb2ExL1KdyEiIZJjth4KItUq\nLCdBJETKrl5C4l6luxCREMkxWw8FkWoVlpMgEiJlVy8hca/SXYhIiOSYrYeCSLUKy0kQCZGy\nq5eQuFfpLkQkRHLM1kPpXiS3IcpEmswf+F2mNskNkRApu7qVpC9kfwGoF+nFNhmYfPkJIokX\nJ4oU/u0G5+HnNkShSGbrEcm1T3ddhEjy6ptEfruhf5Em4/EwIo3zsy5J0SxS5FvguhbpdVSZ\nv71zH+BL9NPOIlSLNNDPugg7xfhZae9BsI2wWrs4yeYYu+csjbJHpMSzCM0iJb0kjCGScfh9\n7Akc21EyIi2PBhIpsc8Ui5T2koBIsZq5hYKj7fXwLUWyTmL3ebRCssFmQaSbHaUiDXhqN86I\nlPaSYL8mPBov2VsH5Q1EWv5HJHuf7rpIq0gulP3xxYjU8NRuxC/RRyRn4xWINOidDf4YaztL\nI0RKJFEuUkpWAESqEERCJESqkDKRhrizAZFudhApOdkiJZHqF2mMOxuSOuUFgEgVgkhFKPvj\nC5EQaQkiiVH2xxciIdISRBKj7I8vRDpcJCNnacSX6Bei7I8vRGJEWoJIYpT98YVIiLQEkcQo\n++MLkRBpCSKJUfbHFyIh0hJEEqPsjy9EQqQliCRGMfajqAmI5CssJrkh0tIS5x5HpNDiEpIV\n4GYHkZKDSEUoH5HWKxbpM8/GV6t2Tozdc7A9RhCpCOUj0nrFIhkANzuMSMlBpCKUj0jrEUla\nGGyr/Z3fI/0L2WpZWnLoRmvlo9/WdyWSMfH9o/l3EIkRKXMxI9JnOhBp3UFrsxFJjPIRaT0i\nSQtDTZ3MqVekg61xxCZBJDHKR6T1iCQtDDXV/s5vQyTjK9IOtsYRu92IJEb5iLQekaSFoaaK\nvliMEWnTit0eR6TQ4hKSFeBmR5lIzyASIolbiEj+IBIiiVuISM5wareQIlLeYkR6ZjL+RyTf\nPt22YrfHESm0uIRkBbjZUSaS6MuXEWnTit0eR6TQ4hKSFeBmR5tIEhJE2rRit8cRKbS4hGQF\nuNlBpKzYJIgkRvmItB6RpIXFJIi0acVujyNSaHEJyQpws4NIWbFJEEmM8hFpPSJJC4tJEGnT\nit0eR6TQYmd8/3pn8B8aO9gaR2ySBJGEfdaBSDKSPkQyJrGPMbeN920HkUSxSVJEkvVZDyKJ\nSBApVjO3UHrEOWK28dTYJIjUtUi+f70zrEhzDrbGEbtFcpGkfaZfJCFJFyJ5/vXOivLX+q94\nnAfAPsJqr7rCqnVi98VpsTslQaRon5n/8sqdpRVpO692hCQ2yoeO1m/iQzhqRPJU5dSuQp91\nMiLFSXoYkRYeRDo6Nkni5e8hRBKSIFKsJiKtDUOkrkXi1O6s2CSc2nUvUuTt3pNkBbjZQaSs\n2CRpIkn6rA+RBCQ9iOT9TPmgOxs8VREpFGGf6RdJSNKFSLIYADc7iJQVm4R77cQoH5HWI1Ko\nJiKlkiJS3mJE+ox5ejDd3SepiLRpxW6PI1JocQnJCnCzo0yk1xvVyZo3SRBp04rdHkek0OIS\nkhXgZgeRsmKTIJIY5SPSekQK1Ux6jzSt7iCSa59uW7Hb44gUWlxCsgLc7GgVyboh8s6X6Nt9\n9IFIWdVLSFaAmx2FIrk+5NuQMCJtWrHb44gUWlxCsgLc7CgVaXmASK59um3Fbo8jUmhxCckK\ncLOjT6TJfIRIrn26awoiyauXkKwANzvqRJrWv4jk2ae7piCSvHoJyQpws6NNJOOyt3WxYSVB\nJLspXYr02o+I5KlaINLk+4onkwSR7KYgkrx6CckKcLOjTCQRyTuIFMymKYduOSfjiPQZ4V5P\n6ZzD+/F9RBpoRLKyaXy91R4RY+/f7DAiZcUmQSQxyn4/djMiGQA3O4iUFZsEkcQo+/2ISJ6q\niFQeRBK2EJHySRDJbgoiyauXkKwANzuIlBWbBJHEKPv9iEieqohUHrlIjyCSvHoJyQpws9OX\nSHNaaxKP3SJEEqPsGo9IjEivIJIYZb8vEclTFZHKg0jCFiJSPgki2UEkefUSkhXgZgeRsmKT\nnCzS2iTH/tcuktV6RArVRKRUUkSStRCR8kkQyd8kx/5HpM18CckKcLODSFmxSRSItLYLkUKL\nS0hWgJsdRMqKTYJIYpTYTkWkUE1EiibjZ1087TpbJOHPunh2qiqR9P0+kvPwyyTRL9KXv3+m\ndVjeD4152naySNIfGvPtWEUiKfzFPufhl0miX6TL5TJ9/7eoz/JFWtvn7A7dIu1aj0je1RaT\n6Bfp9z/f/rh0+frPr2zSMpEcjfX0jkaRtq1HJM9qi0n0i/TIvz+mPy59kY5L7m857zEWyRgo\n45DoPrxcFxt+/bg8hyXZGlJGpMis7vdIpW+CTh+RYhfTDissJ9HCsrZzL9J/357D0c+vl29Z\npIiUtxiREki0sKzttEX69+vrrO4iuzSOSHUWI1ICiRaWtZ325e/L5dt/y6LpLgki1VmMSAkk\nWljWdtqXv3/8d08MItVZjEgJJFpY1nbal7+TumtG3d4PEL6wElxa8NTIUlksktRtlC0uffo2\nThT5Ks6tuYmnU7Sx2O+Dvj0LLl/knyOltUa1SLF0JFLhKhSJdOB2K4r04/MKw+XyXbzOtNYg\nUqunC6Lt4CuJNhZLpOnyvNfuP+EVu/TWIFKrpwui7eAriTaW3b122ykhRBJLmG+X77/v998/\npLc1EEIesUT6NT3vDrpMyVfBCXnn2Kdwv398uVy+/Mi/aEfIO4b3QoRUCCIRUiG2SD/mN0nC\nG1aNn3B3/aS7bxp54vIT8eb0tXQKLnVtrFKM9UW4J8fUXuxd/iIzUCtBSmvKV9pk84Xb3R9h\nrmnCwSqquf9ANkGk131Q03Y+No09cWneZ9HkWM1k1opsrFKM9Qm5Z0n8u2NylS/U9+20AqTY\nDvFKm2y+cLvTZNbw7zDxwSqruftA9n85PEeLNG1qxXZEndQVafKINLUTaZJXlK60yebLtjsZ\nC08UKeOD2GlFkrct+MTJnHpFWmtJNlYee31+7smchkRytrjdiJS0N+R1ZTWntM0LNulbnVCk\ne6RGYs3dB7LJ938/D+ppeXVNFMn5xNd7H59I07aWaEcUxyWSm3t+gxMSyXgrhEjp0S/Sr+lr\n4kdIKQJt2uZ/4ipQUCTfUveOKM5+x/o2+WpfTKQg/bkiSatKLza8dleV1BJJcvA+J/FxYv+9\ndilX7axGJYsUeGJUpLtvqXOXlcchUoj7TUQSVq3cGUeLFK5RItLr0qB8S7bkkSeGRLJqiXZE\ncfZdEOQOihSyQ4VIKftNJNJ6Ib9GKomUoFyqSFlEYoEm6ymRQyks0qmndkHu+KmddpGkFVt5\nXLRdsR6yg1d4WJeJZLRo+V/W7cEnTnfjulxQpM9a8V1WI/su8HFPswESkVRebEhbYa8iTdvq\ntUX637c/p3VfZTd/bz6ivydOA09cr8itU6NWeKnjk+k62X7SHeKeHFPx7nmRnXZnQ8JpWIvN\nl2532tTw7jDxwSvrN/vLT7483x9dLsk/SkHIO8cS6fvlx+ND2X/4h32EpMRxZ8PyPyFEGkQi\npELcp3Y/Cr6Oi5A3jH2xYfnOBv6tOSEJ2Z3C/f38zoacry4m5H3DeyFCKgSRCKmQ4ru/CSGI\nREiVOIX59fXvo9tBSNdxjzy/L5hESEI8p3D5p3Y/lSWXQx8KJJpJ3ML8I/whZv2kQ3YaJFqy\nNsx3seHHIKRDdhokWrI2zC3SlO2RNtIhOw0SLVkbVv0y99lodsZBgUQzCSJ1gwKJZhLvB7K5\nH8qejWZnxE6DREvWhiFSNyiQaCbZ/T7S4wuEfn1N/eJitaRDdhokWrI2zPMl+vlffnI2mp1c\nDn0okGgmcf+sy2/ubNCHAolmEkuYr5fHSd2vr5dvg5AO2WmQaMnaMEuk/4q/s0HWgOs63Ty8\nLllrmvPXzZMTURugXDfTq9ngq/VwW39B2uGeRbJH2+33LeyrUsa6RySxT+F+/3h8Z0PBvd9C\nuHV6XY4/tyfbHXDdVxCjNkAx2r5rnf0asdYw/qagNCXZoxmTDdnVqpNx/I1IctIHsm8tkqPG\n6SR7NGNivgBc7To9itSARIFIM1WXIq1tzxApEaV5p1ho62R8kcyKeSRFv0aRT5oq0qvLUs+H\njhPJbt1eJEf7uxBpbfN12+DuRPKSmBXzSKr/GkUSnESk9a1iJyJ5LzZs2v9Z3sXFBmOfO17k\n9IskIMkE8YtU/msUQrh1urgUPbW7bh8kozZAcbXdOLq2D3ftT0RpSuJC2zzy9o1+kQQkmSDh\nD2TLvkQ/DS5JpOWVRadI29Y5dNq1vxeRjEGzc5G8JHkcvYq0fZCM2gDFOyI5RbLb34tIu7+b\nJT2J5CXJ9Ch6alfwaxRpcNfNO4egSOtTVIk0/79rnXU6frWRNzXOJ3GibR44rOlMpABJpkeB\niw1H3dkwD7ICkfYnRAlv0M8TaTf0mN3Yw50N9gPn4dfFnQ0CkpQjyk1S/dcoMkibJhtEHQok\nmkn4p+bdoECimcS++7v4l/oqtTDlnCeY81FqRQ1Jcd+MSGKJNBWPUBUxq2QcFEg0k9j/jOLr\nj8IfvTwbzc44KJBoJqn+sy5no9nJ5dCHAolmEkTqBgUSzSRctesGBRLNJNVFui1ZHzWZl1Yv\nRoltKBkkl9xq3efvhUyPzNO7Y2p1iq9JwqLc51lFxX1SvUWeogQSU6Qqv3fpbQQi5T7BKdLs\nyWzLNP+xp3an+JqESM6iEpFKbfI2ApFyn+ASabojUsUWeYoQaXiRLFcQqUkRIr2NSMtbpLlk\nL9JfjxTsBGWJ7cu6RYj0NiLNfxiRmhQpEenjEW+j3kSk7U5Iaro1b7VuMh8ViuRopKNWFyIJ\nSRAptXoxCiKFShBpzlakwt9G2pAi0jNtRap0aneuSFU/ERtGJCPP7qmwnopJ6bNeRApfbDBI\n/Fs4VaS6n4hpEKlKjC3qG5GS+qwHkbyvBWmv42eKVPkTMUQ6QKS0PlMuUlaneLag4NSu1idi\niHTIe6SUPkMkQUk9kaKfiN1Fn4id9XbiLUWS9VmdjzFVvFG8RXaWBpHmP4xINqlikWR9xogk\nKKkm0vIIkWxSvSIJ+wyRBCWINAeREEmFSJza+fpMr0ic2ikVKfLGVUKCSAeLJOgzRBKU1Du1\nq/OJGCIdeWon6wMT3/gAABTcSURBVDNEEpTUEKkmCSIdI1IGCiKFShBpDiLFUBApVIJIcxJE\nSjyJRaRnEGlf1JCkA5FSr0+OIpJ9kCFScVFDEkRCpNh8AUksz0Y2XH+9xHZ+LyLdEalHkWJN\nZkRyFzUSKfX23H5e59xZewaRfCWINCdBpNkiRqSkhiLSvqghSQ8izX8QKamhiLQvakiCSIgU\nmy8m8W8BkdxFiIRIDhL/FhDJXYRIiOQg8W+hH5GiOesCF3c2xFJ47VHFpctbZGf1I1J0W/pH\npFTS/kX6XAEjUqgEkeYgUgwFkUIliDQHkWIoiBQqQaQ5iBRDQaRQCSLNQaQYCiKFShBpDiLF\nUBApVIJIcxAphoJIoRJEmoNIMRStItX87h1Echch0vgiTSlf0RdrMiK5i84Uycizexquv3XW\nntEn0lT1+0kRyV3EiDT+iIRI7m0hEiJF5q3WCUUS/UBNP+cOsZ2PSIgUm7daN86IVPOyCSIh\nUmzeat0wIlW9bIJIx4iU8uKHSIKScpHqXjZBpENESnrxQyRBSbVTO0SyYmxRnUhpL36IJCg5\nUiR+jNm5e887tRtIpCHubGBEcsXYYsciGReNCy8Nq7iyfIvsLERyFyHSG4xIWZ3i2QIiuYsQ\nCZHcneLZAiK5ixAJkdyd4tkCIrmLmomU8r4WkZxPT2i6NZ/UU9vEmqxApLe6syH2axQbUkRy\nPj2h6dZ8Sk9ZiTX5XJFqknQh0ufHMP2LNMadDSmJNRmR3EVtRJql6VqkjE5DpFAJIs2pKhI/\nNLafR6R9UUOSDkSa7oxIrg0hUnJRQxL9Ir18QaSkhiLSvqghSQcifQaREKm4qCGJfpGeYURC\npApFDUkQCZFi88Uk/i0gkruoqUhd39mQ0WkjiBRLP9dXYzu/E5FSSBHJ+fSEplvzNTrFswVG\nJHcRIiGSu1M8W0AkdxEiIZK7Uzxb6EekaM46SUWkWArfP6h4+3GL7Kx+RIpua/gRaddX+kVy\nN5wRqbioIQkiIVJsvkaneLaASO4iREIkd6d4toBI7iJNIi1giJTcdGu+Rqd4toBI7iJEQiR3\np3i2gEjuojNFMvIS6TXTcGMNsvYMIvlKEGnO8SPSMmVESpyv0SmeLSCSuwiREMndKZ4tIJK7\nCJEQyd0pni0gkrsIkRDJ3SnrGjft6lukTeMRCZEi8zU6ZV0jIgmKEOltRJr//X/695MikqAI\nkd5HJGOS9I3ZiCQoQiREuvcmUvbYikjhRjcUSdZnPYg0mdPORcolQaRwo1uKJOqzLkRaXhK8\nKH95fnn12S7nzElBpG2MLSKS4+kJTbfmvSQJXza4rlHbiJQ/tr6RSDujXI08TiRhn/Ug0sIx\ngEjRsdX7q+ab4fSssTXlS/TTfgnKJZIp1Ikiyfqsmx9jHkOkbJLeRqRUUsUiyUh6GJHGObW7\nWy1HpHWLWkUS9lkvIkUG102nrGtEJEFRu/dIiKRLpPxfXtUm0vuc2i248neDfpHSNlo7A53a\nCeNoskaR8sbWHkVKIdU7Ign7DJEEJfVO7fJ/1RyRgo1ueGrHjzErFCmbpEORJvNPxyIlddrb\nieTtoc5F2i45U6Rp/YtI0pYg0r6onOTV3h5FmoxJoUiuUwlEiswXk2zWiEiCdjcRaXrdEFB+\nZwMiZcwXk2zWiEiCdrd6j5RAKhApcoAikjVfo1PWNSKSoN2IhEjuTlnXiEiCdiMSIrk7ZV0j\nIgnajUiI5O6UdY2IJGg3IiFSkASRhhUp+XBDpAISuUjb5iMSImWjjCDSZ55NKZlREGOvvvYN\nIhkyaRTpM4WHkooj8fbax72PSHM2O9U/439O9SBSDOXDalqPI5LdlH5FMtr4WvGmifvdvWx+\nu0T9iGREKNKrbsOmZGXtGUTysCPSEkakGAoiIZKABJFiKIiESAISRIqhIBIiCUjUirQrQKR6\nJIiESD5GREogQSRE8jEiUgIJIo0o0tYo7w5BpHokiDS0SOEdgkj1SBCpG5FyJEKk4HxFEkQa\nXqRXxyGSPV+RBJEQyceISAkkiIRIPkZESiBBJI0iOb+Oq0gkUygfVAuRBvrKYlen9CmS9/B6\nrXgQkdxfENmjSBq/RP/x7AyRnJ1SLpLjKG0tkv/weq0YkRApOo9ILpJykVzVxhXp1YvG4eRj\nRKRop1QW6XNmBJHmGT0i+X52upNYJGOgjEOi+/BqNSJlv6WWzh82IsU2lAxyxMUGwe/c5RSd\ncLGhEUk375EQqd4TEAmRAo1CpMT5Gp3iaxIiOYsQCZHcneJrEiI5ixAJkdyd4msSIjmLDhXJ\nvh9gSewCS+Hy0tW74iSJrqi4JYehFG9E1pK6l9bakNTkfabd99p1KFLeilSKlBy1IiUHkZKW\nI1LdIFL2qp5BpFgQKbESItUNIh22ghpBpOxVPdPyu78JeZsgEiEVgkiEVAgiEVIhiERIhbQS\nyfw8enk8zQ/M+Vj5vHTyVpvmCmv2q9hPZQxOGm8Fu4ZrFwQrJK+gXkSrl7WhcUvrbL4i75xG\nIpl3SL0eW/Ov6WTNO6bTOr9dvLi1/h9blXk3XZhhPYrdTzN2s2Odzl3gq5C1gnoRrV7WhsYt\nrbP5irxL+hBpMuatxc1EmtaK7qcZFRApsVazjCzSUjBZyxKm02LMgSIZFX1Pm3YPPDX82502\nk4wVVIl89bJj9Lwk7CiRSOIcKdJyHrbMm+XBo//zz7OWUpF8p9NikbJXUCVVRTr1PVJVkZS9\nR1oeZoxEW5HuT1NUihSp4N1uuMLakTpEkr1DF62qVaQ7SoCSRnKgSMuDTJHuikVyzG2KoiL5\nVtDjiCRdVZNUHZHEq7ofJdLmUbJIhh19iVRuIiIlZmyRpk1Zpkj9ndpZLyQB0ZwVzDINIsna\n0IVIFXmXHCGScTBMxjJzGrzYsIqk9mKDa/lkPYxVsJcjUkZGE8n8VHhza8I9f3rSnQ3mJlMq\nrNieFYgqRFpQK9zZkL2qOdxrR0iFIBIhFYJIhFQIIhFSIYhESIUgEiEVgkiEVAgiEVIhiERI\nhSASIRWCSIRUCCIRUiGIREiFVBfpp7KMgwKJZhJE6gYFEs0kiNQNCiSaSRCpGxRINJMgUjco\nkGgmQaRuUCDRTIJI3aBAopkEkbpBgUQzCSJ1gwKJZhJE6gYFEs0kiNQNCiSaSRCpGxRINJNo\nEOn6mXnm9ceYzpXKUA9AEZHs5jSSyDslg0UpSca614apEGnDaKBerTplqAegSEgecz2IJEHJ\n4tBIknt4rQ1TKtLz767LehRpR7Iu0UwiQ+lVpGqH19qwXkQyK2aiHoAiPfoGESnToyFJlIm0\nArpetnsSyU/SnUhelMy3SBpJfg7xHml9N2i8UthcnV1s8JI4x1p1JGKUshMiNSSZvbI2TIVI\n1iPX+3NfUQLqASgCkuw3e+e8jodQ7Iodkwz1Hunn5sWjd5F8JJuLsXpJmnaKPpLhRNr9LSVV\nNyJ5SrSRHPQ6Pg6JKpGu5rRrkQIkuSBnHX6RTlHfJ00Pr7VhvYjUyZ0NEhJniTaSpp0yIokG\nkZpmHBRINJMgUjcokGgmUS7SNf8i1xwtKOOQlKOMSKJcpPKMgwKJZhJE6gYFEs0kiNQNCiSa\nSRCpGxRINJMgUjcokGgmqS7Szcqu4OCipijjkMiKxiGpxLs2DJFKUMYhQSREkqI2QOmFZJwc\n2wOI5EJtgDIOCSMSIklRG6CMQ4JIiCRFbYAyDgkidSfSx59UIz210z5cKOOQ9ClS1cMLkVyo\n1VEQSR+JbpGmz7+PzNO7Y3oGqb7DbxwSRKot0uzJbMs0/7Gnp5DqO/zGIUGkyiJNd0RybguR\n9JFoFslyBZFMEkTSRdKDSMtbpLnEIdJffxJd2YM0ad+dlthuRaQ6JHrfgrcbkRwCdTUipfQZ\nIh1DovgteBORlkeqSBM7LanPEOkQEs1vwRHJhZraZ4h0EInet+Cc2rlQU/tMuUg131loEKnz\nt+BpIkVIH4nuVQUiyfos2mkfZ143qfrOQoNIHoLxRiTvi15Pdzak9JnqEanuOwsFIt09BEOJ\nlJJjSbNEEvaZapHqvrNAJHcRIrlQU/usC5GiZ6n6T1Lf6NQuJceSvu2pXd3DT8GINBn/I9Lh\npNkiCfqsB5HuHoT+RFL5FhyRXKipfYZIB5HoPbwQyYVaHUWBSEOc2tXsE0RqVdQQRYlIFd5Z\nIJK7CJFcqNVRFIhU6Z0FIrmLEMmFWh2Fe+30kSBSq6KGKIikjwSRWhU1REEkfSSI1KqoIQoi\n6SMZR6RoTr0bJSmx3YpI+kjGEelYUn2H3zgkiIRITtTqKIikjwSRWhU1REGk40miOeudAyLl\n56Obt3uxvdaPSNFtMSI1KmqIwoikjwSRWhU1ROlHpFjGGVsRqVVRQ5R+RMoiYURCJCdqdRRE\n0keCSK2KGqIgkj4SRGpV1BAFkfSRIFKrooYoiKSPBJFaFTVEQSR9JIjUqqghCiLpI0GkVkUN\nURBJH4lukfR+8Zi+w28cEkSqLdKk9ztl9R1+45AgUmWRJsVfzqzv8BuH5DCR9J7wNDm1G0Kk\ngb5pNSVZJEeJpPiE5yyRVP+k2iNJfYZIh5BoPuFhRHKhpvaZcpHG+YJIvYcXIrlQU0l0i1T1\nhKgPkVSf8CCSt89U/zxX3ROiPkSSkDAitSrKJ9E9IiGSe1uI1Kgon2QMkdSPrZoPryYiqbzQ\nj0hVDj9GJHdRbZFSciwpIolQskgQCZGcqKl9hkhHkqg84UEkF2pqnyHSQSR6Dy9EcqFWR0Gk\nOiQ1+wSRWhU1RFEg0hB3NtTsE0RqVdQQhXvt9JEgUquihiiIpI8EkVoVNURBJH0kiNSqqCEK\nIukjQaRWRQ1REEkfCSK1KmqIgkj6SBCpVVFDFETSRzKOSNE8SA/faFZiuxWR9JGMI9KxpPoO\nv3FIVIoUzVmv04iUn49uBtfYXutHpOi2GJEaFTVEYUTSR4JIrYoaovQjUizjjK2I1KqoIUo/\nImWRMCIhkhO1Ogoi6SNBpFZFDVEQSR8JIrUqaoiCSPpIEKlVUUMURNJHgkitihqiIJI+krFF\n2sAhUickiIRITtQqKGYQSR8JIrUqqoxiBpH0kfQg0jR/M036F9YgUo8k3YjU7vBqJJIxSfsK\ntXFE2hxxiKSDBJHKSfM6TTa2ItKhJBpPeJqINJnTzkUSkXQhUvbhFyc5WiSFh1cbkZYu85P6\nfornAeeeOScjiSRCySJBpIYjUt7XTCsbkYRjKyIdSaLyhKfNeyQPYYciycZW1+D6YY6nm5lz\nkn/4qRMp6YTHf45zVp+8pUgykh5GpPjh19VLgvjweu35DkekcU7t7lbLexYpu1O0jUjCTjEa\n/FpxhyJFxl5PnyFSUxJE6kukgp/iUSbSQKd2C4/o8PswDz9tIr3PqZ0wx5JmiyQYW/MOP8Wn\ndupFkp/wIJIGkdJ+Q1a9SOLDT7VIaSc8iBQi3S5pfUIkRVEtUtLhp1uklD5BpCApIqkiQSRE\ncqJmoiCSPhJECpIikioSREIkJ2omCiLpI0GkICkiqSJBJERyomai5Ii0XdIpCSIhkhM1EwWR\nyprdggSRgqSIpIoEkRDJiZqJgkhlzW5BctThdaZIcx6tFsz4n6Mga88gki6SdxDpWNKmnfaZ\nD9Nv/4z/OQqy7vlUkbS9JMzR9zqNSDGUEUakz4zzksCIVJsUkdqRqByRjAZv2vhaNyIhkjIS\nREIkJ2omCiLpI0Gk2qT6Dj9EQqQqOZZU3+GHSIhUJceS6jv8EAmRquRYUn2HHyIhUpUcS6rv\n8EMkRKqSY0n1HX6IhEhVciypvsMPkRCpSo4l1Xf4IRIiVcmxpPoOP0RCpMxU+yrM00Wq9k2r\n5x9+tb4gUifJkCLV+3Lms0Wq993fpx9+1b6yWCcJIslJvQdpNmoqCSKpJUEkOek4IrmqdSqS\nFpKjDq+zRPL9qnknsUjGQBmHRPfh1WpEkkvduCifRICilaRVp4xD0s2p3UGkiHRop4xDgkj5\nqKkkiDQwCSLlo6aSINLAJIiUj5pKgkgDkygUyb4fIB7ZpZeatYQZh6QRyjgkNXmfafVNq6Ho\nPfxSMw6JXpGSg0jtarXKOCSIlL+qZxCpJOOQIFL+qp5BpJKMQ4JI+at65gyRCBkuiERIhSAS\nIRWCSIRUCCIRUiHHiyT7pLpmrVYZh0S2+XFIavLOOVwk+4a29rVaZRwS2ebHIanJuwSRsjMO\nCSLlr2qJUpHWqkUVmmYckgSUcUhkIonTuUhnno+PQ1JXpE5IBCIN8B7pLoJ4VlF+atcDiRRl\nHBIJShqJWpEktXp4j7RWrbSuFql9atcDCe+RstbVIOOQIFLhqu5qRerhCtE4JFy1y1/VEkTK\nzjgkiJS/qiXc2ZCfcUi4syF/VXO4146QCkEkQioEkQipEEQipEIQiZAKQSRCKgSRCKkQRCKk\nQhCJkApBJEIqBJEIqRBEGiEXuvHs0AMjBJFODz0wQhDp9NADneX35ctz+uXy3/3nt8tl+nH/\nFOlTpuff398vl++/T2zkGwaResu3y68/f3/98enfyzM/diJNj+Iv5zbz3YJIveXfhzn3H5d/\n/wxK/9zv/y0SrSL9/ajx4/K/kxv6XkGk7vLl+TVRz3+7+evfv7/uRfry+ejbiW18vyBSd/nf\n5ef95+XvP4++fp7b2SJdLks5OSzs7e7y+/L9z4nb7/v9++XL//79hUgqwt7uL98vv57nbZ9X\n6DYi/VpP7cihYZ/3l59/Rpuf94c6P++/X++Rpss/89yPx8WGfy5fz27nWwWROsyXz2vbPy7m\ne6Tn3N+PR7+fl78v/53dzLcKInWY/z2ue98f53iXrz9fp3U/psvf8wnec8GpTXy7IBIhFYJI\nhFQIIhFSIYhESIUgEiEVgkiEVAgiEVIhiERIhSASIRWCSIRUCCIRUiGIREiFIBIhFYJIhJgR\n//PiS2COkHfPZTMJ1EMkQvwRinRhRCIkEEOk5RTvMTHnNxWdc4S8e1aRLsb0YsxvKzrnCHn3\nWBcbLKEQiRBRTCPmr5aZi63LeYhEiD/WuyB7RHJW3M0R8u6xhhxEIiQnW5GWUzsuNhCSFPPU\nbr3szeVvQmok/gEtISQU65QuUIkQ4o/kJlZEIqRCEImQCkEkQioEkQipEEQipEIQiZAKQSRC\nKgSRCKkQRCKkQv4PPVhvNE0q8p0AAAAASUVORK5CYII=",
      "text/plain": [
       "plot without title"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0gAAANICAMAAADKOT/pAAAAPFBMVEUAAAAaGhozMzNNTU1Z\nWVloaGh8fHyMjIyampqnp6eysrK9vb3Hx8fQ0NDZ2dnh4eHp6enr6+vw8PD////GSW4mAAAA\nCXBIWXMAABJ0AAASdAHeZh94AAAgAElEQVR4nO2dC3ubSBIA5SUv53aTOPr///WsJy1g6Jnu\nRjSo6ruzkxaIKaGyZFkbH44A4Oaw9gIA9gAhAQRASAABEBJAAIQEEAAhAQRASAABEBJAAOEh\n/UrGflQwyWxCSJtRwSSzCSFtRgWTzCaEtBkVTDKbENJmVDDJbEJIm1HBJLMJIW1GBZPMJoS0\nGRVMMpsQ0mZUMMlsQkibUcEkswkhbUYFk8wmq4T0duH0p/P/LsOJ7R4ueLvtd9rzun+L6hIq\nvcltoZM6MyKT4iuYBKgkOSfrmKwT0v3TbEhv11vhaizF6++By6o8ni2xqsEpK4rc/7q2iV8l\nyzlZx4SQXCoT5+zX22hpcyG91WksbuJXyXJO1jHZQkjXW2GkWsWyKm3nbEKkweVpJ8WussWQ\ngkxWDOnqVgzp7f7/qZByPB8X56w/VcMHmTmRKfFVTEJU0oX0PJPELzbMhjT+CqOqLqEyeM1A\nfi2sE6nVWNwkRKXSJb/Jnl5seHvY5u3Xg+rb1B6K6hIqw7vQ6OuaJlKrsbhJiEqSkCJMtvGI\ndP80F9L9tchthCRecx2cszmRWo3FTZ6psgGTPYX0sE3+kEYfq0RqNRY3iVDZi4nl3pU2pH78\ndv1b6pAeLN6mNpkWqdVY3CRApVYkvcl2Qnp4seH6l4efR+snLdWrdoT063Yi92CymXc2PJP9\nqGCS2YSQNqOCSWaTdCG9vdU/RahhNZVokRXvfrs5JwuapAspmv2oYJLZhJA2o4JJZhNC2owK\nJplNCGkzKphkNiGkzahgktmEkDajgklmk/CQPiaYHD5hfrogQGV89atMIk5KzQHrRp79FjGp\nWKLpBMxu0i+MkHQVQiKkwqRfGCHpKhlD6j6p+Tw4KYQUu1O/MELSVRKG1F0/aJ+HJ4WQYnfq\nF0ZIugohEVJh0i+MkHSVhCGdIaTGJRKScU5Ix+M/Jxw3QjLKNxQhLTffc0iXFxN4RGpZIiEZ\n53sO6chTu+YlEpJxTkjDk0JIsTv1CyMkXSVhSLxqt/uQdoTr9iek8n4B52T/IY0OO7mW58z3\n+4jEOxsI6YnzHYdkPCmEFLtTvzBC0lUIiZAKk35hK4b05ZPJC3KF9GVindsMacpkoyHZzgkh\nGeeEpJoQkmunfmGEpKkQUuVCCSmU0WEn13KCkAjJaEJIEkIiJKMJIUkIiZCMJoQkISRCMpoQ\nkoSQCMloQkgSQiIkowkhSQiJkIwmhCQhJEIymhCShJAIyWhCSBJCIiSjCSFJCImQjCaEJCEk\nQjKaEJKEkAjJaEJIEkIiJKMJIUkIiZCMJoQkISRCMpoQkoSQCMloQkgSQiIkowkhSQiJkIwm\nhCQhJEIymhCShJAIyWhCSBJCIiSjCSFJCImQjCaEJCEkQjKaEJKEkAjJaEJIEkIiJKMJIUkI\niZCMJoQkISRCMpoQkoSQCMloQkgSQiIkowkhSdKHdOF89wu4Ydakv8EJKXSnfmGEpKnwiFS5\nUEJSufx+7O76m7Irf4H27FpOEBIhWcn3LKEmpGsn11q664fh5yvjG5OQYiaLmGwtpI9epX2J\nKz8idUdCUg9ISISk89AKIRGScz+3ybZDun2LdJ1MhPTPJ/W3Sb7nuQP6c0ZIto0I6ZFOfuAR\niZCc+7lNNh3S7U+EREjO/dwmhCQhpKeFdGE/P1redEg8tZs7YO6QyiY8Irl26hfWFpLyYoMw\nVdZygpAIyWiy6ZCK72jgnQ3PmSxiQkiunfqF8V47TYWQKhdKSKGMDju5lhOEREhGE0KSEBIh\nGU0ISUJIhGQ0ISQJIRGS0YSQJIRESEYTQpIQEiEZTQhJQkiEZDQhJAkhEZLRhJAkhERIRhNC\nkhASIRlNCElCSIRkNCEkCSERktGEkCSEREhGE0KSEBIhGU0ISUJIhGQ0ISQJIRGS0YSQJIRE\nSEYTQpIQEiEZTQhJQkiEZDQhJAkhEZLRhJAkhERIRhNCkhASIRlNCElCSIRkNCEkCSERktGE\nkCSEREhGE0KSEBIhGU0ISUJIhGQ0ISQJIRGS0YSQJIRESEYTQpIQEiEZTQhJQkiEZDQhJAkh\nEZLRhJAkhERIRhNCkhASIRlNCElCSIRkNHmFkOo53RjPPaL266Qff600IbUtlJBCGR12ci0n\nnv6I1F0/aJ8fVQipcqHGkNq+un30Ku1LJCTjnJBUk9VDajwpH71K+xIJyTif+h6JkLQDElL9\nJr0HIU2es39OXHb48vzv5cLpb/B0IZ0hpNI509ZyYpWQumP1OeMRqW2hi4fUf3Fb4YUqDUIi\nJNMoMqSGk/LRq7QvkUck43wcUic/bDkky2tdhGTxIqSPcUhd/3HbIZm+Rc8aUstXt49epX2J\nhGScD0PqHk8cIU0e8NkhNX11++hV2pdISMb5IKSuuz7h2ck7G/YRUttXt49epX2JhGScj79H\nMqhsPKTsL+Q3fnX76E9K+21JSMb5vkNq/hY95yOS5ZwQkoSQCMl4TghJQkiukNpf6yIkixch\nfew6JMNrXYRk8SKkjz2HZHmti5AsXoT0seOQTK91EZLFi5A+dhxSmAkhuXbqF0ZImgohVS6U\nkEIZHXZyLScIiZCMJoQkISRCMpoQkoSQCMloQkgSQiIkowkhSQiJkIwmhCQhJEIymhCShJAI\nyWhCSBJCIiSjCSFJCImQjCaEJCEkQjKaEJKEkAjJaEJIEkIiJKMJIUkIiZCMJoQkISRCMpoQ\nkoSQCMloQkgSQiIkowkhSQiJkIwmhCQhJEIymhCShJAIyWiy1ZCu/4Zayz/OrKzlBCERktFk\noyF17f+op7aWE4RESEaTbYbUWX4TlLKWE4RESEaTbYZk+WemtbWcICRCMprsPSTxa6crON0Y\n9VuvQH/OCMm2ESE9wiOSekBCIiQdQlIPSEiEpENI6gEJiZB0CEk9ICERkg4hqQckJELS4Z0N\n6gEJiZBiGR12ci0nCImQjCaEJCEkQjKaEJKEkAjJaEJIEkIiJKMJIUkIiZCMJoQkISRCMpoQ\nkiR9SBfOd7+AG2ZN+huckEJ36hdGSJoKj0iVCyWkUEaHnVzLCUIiJCv5niUQkqZCSJUL5REp\nlNFhJ9dygpAIyWhCSBJCIiSjCSFJCOlpIV3Yz+uPhCQhJB6RjCaEJCEkQjKaEJKEkAjJaEJI\nEkIiJKMJIUkIiZCMJoQkISRCMpoQkoSQCMloQkgSQiIkowkhSQiJkIwmhCQhJEIymhCShJAI\nyWhCSBJCIiSjCSFJCImQjCaEJCEkQjKaEJKEkAjJaEJIEkIiJKMJIUkIiZCMJoQkISRCMpoQ\nkoSQCMloQkgSQiIkowkhSQiJkIwmhCQhJEIymhCShJAIyWjyCiHVc7ox1jt6Bf05IyTbRoTk\nNlXWcoJHJEIymhCShJAIyWhCSBJCIiSjCSFJCImQjCaEJCEkQjKaEJKEkAjJaEJIEkIiJKMJ\nIUkIiZCMJoQk+XKldntCWsCEkFw79QsjJE2FkCoXSkihjA47uZYThERIRhNCkhASIRlNCElC\nSIRkNCEkCSERktGEkCSEREhGE0KSEBIhGU0ISUJIhGQ0ISQJIRGS0YSQJGuE1F0+fjL3+UGF\nkCoXSkihjA47uZYTK4R07eX6ofT5UYWQKhdqDqnlq9tHr9K+REIyzochdUdC0g/47JCavrp9\n9CrtSyQk47zw1I6QZg/45JDavrp99CrtSyQk49we0j8nLruc1xh9KwXQ/oQoZ0htX90+epX2\nJRKScb7nRyTDE6Kth9R/cUv4byIS0jZDsjwh2npI4u7FI5KAkFzfIxFS8xIJyTgnpO18t0dI\nA0aHnVzLCULiEcloQkiS1ULaxzsbCKl5iYRknI9DMqgQUuVCfSHxzoYBo8NOruUEIRGS8ZwQ\nkoSQCMl4TghJQkj+kHbxzob2c0JIEkJyhRRjQkiunfqFEZKmQkiVCyWkUEaHnVzLCUIiJKMJ\nIUkIiZCMJoQkISRCMppsO6Tu+kpQ1U/MlLWcICRCMppsPCTxSX0Ph7KWE4RESEYTQpIQEiEZ\nTTYdUic/E9LzJ4uYEJJrp35hDSHdvkU6Hkshif8WuIJbSPV7PJn+nBGSbSNCmqDtfe7KWk7w\niERIRpNNh3SGkAoHJCRCaoGQCgckJEKqhKd2cwckJEKqpBP/JyR9CYRU3M9tsumQ2v5bYGUt\nJwiJkIwm2w6pyVRZywlCIiSjCSFJCImQjCaEJCEkQjKaEJKEkAjJaEJIEkIiJKMJIUkIiZCM\nJoQkISRCMpoQkoSQCMloQkgSQiIkowkhSQiJkIwmhCQhJEIymhCShJAIyWhCSBJCIiSjCSFJ\nCImQjCaEJCEkQjKaEJKEkAjJaEJIEkIiJKMJIUkIiZCMJoQkISRCMpoQkoSQCMloQkgSQiIk\nowkhSQiJkIwmhCQhJEIymhCShJAIyWhCSBJCIiSjCSFJCImQjCaEJEkf0oXzGgNumDURN3jF\nbUdI1Zv0CyMkTYVHpMqFPiWkK/m+uBGSprKHkC7s57GVRyQJIfGIZDQhJAkhEZLRhJAkhERI\nRhNCkhASIRlNCEnyRVKxPSEtYEJIrp36hRGSpkJIlQslpFBGh51cywlCIiSjySuEVM9DSOst\no0x/zgjJthEhuU2VtZzgEYmQjCaEJCEkQjKaEJKEkAjJaEJIEkIiJKMJIUkIiZCMJoQkISRC\nMpoQkoSQCMloQkgSQiIkowkhSQiJkIwmhCQhJEIymhCShJAIyWhCSBJCIiSjCSFJCImQjCaE\nJCEkQjKaEJKEkAjJaEJIEkIiJKMJIUkIiZCMJoQkISRCMpoQkuTLEGV7QlrAhJBcO/ULIyRN\nhZAqF0pIoYwOO7mWE4RESEYTQpIQEiEZTQhJQkiEZDQhJAkhEZLRhJAkhERIRhNCkhASIRlN\nCElCSIRkNCEkCSERktGEkCSjkO63Tr6QBieOkAjp8sc7hKSpEFLlQgkplNFhJ9dygpAIyWjy\nYTsnhDR7PYTUYqIfkJDqN+kXRkiaCiFVLpSQQhkddnItJwiJkIwmhCQhJEIymhCShJDWCGkY\nEyF5duoXli2kM4Q0NVnEhJBcO/ULIyRNhZAqF0pITrpPRqbKWk7kC+nRZNMhTZ2UbYY0effa\nY0jd/YM0VdZyohzS8GTPX09YSAOT4d1PrCp9SJMnZZMhTd+9CEkwH9JUTITkOimTt+22QxIq\nhFTB7PU8M6TranYQ0u123UFI1V93NxPSP5+4r289Bib7UNmPSe6714qPSMvPQx6RVnr8GU8i\nTkrsI4t1v0VMXvepHSERUqAJIS03J6ThSSGk2J36dRJS0YSQ2hZKSD4I6UmTiJNCSLE79esc\nhPT1568Wy4uq8Z0N6UKafmfDJkMqnZTthVRtkiukw+HQ/fivSXTA6LCTa3nOvDWkgsoWQyqc\nlA2GVGuSK6S//37/bOnw7d8/btPZtTxnTkjDk0JIsTv1C5v4Hum/9+6zpa/Gx6XRYSfX8pw5\nIQ1PCiHF7tQvbOrFhj/vh/PDks90di3PmRPS8KQQUuxO/cLGIf3+fn44+vXt8N1lOruW58wJ\naXhSCCl2p35hw5D++3Z/VncwvTQ+OuzkWp4zJ6ThSSGk2J36hQ1f/j4cvv++XdQdDYwOO7mW\n58wJaXhSCCl2p35hw5e/338ffYwOO7mW58wJaXhSCCl2p35hw5e/HY7ttL4pvvlN9DHvuq+5\nlmdu46Dq6uM2WlCn4pqftsmJ4fdB38+Dw1fzz5FaICTTNg4IaYlNTgxCer+8wnA4/Kjb3Qch\nmbZxQEhLbHJiEFJ3OL/X7rftFbtWCMm0jQNCWmKTE6P32j1+BoAaBsF8P/z4ezz+fTe+rQHg\nRRmE9Kc7vzvo0HlfBQd4KYZP4f6+fz0cvr4/5UU7gN3A90IAARASQADDkN6v3yTFBzb477jH\nFww26Eo7yO0fNyi8ObBwgEqGew0PP7z2bsDUCobblK7HvuoqlVmHKVdlw5tL3daBJo3npKjo\nOCPjH8guFNLwXxYZXTDcoHRT37cbbVAKtXCAOiaXJQ4/vHb5uevKl50/V15P+6qrVGqOfQ3j\nKqOI3O5niopfaXANNefkuvzu5jWztuv9qG35ox/I/s8jOMPE4afX+3hpefvRBt3EZO4AtnWH\nhnSsu571QrrEoYXU7Saku+nsrTK1/MIPZOPpZe8PjLP38/ufZrefOEbLDi3rHgy7weUTJ60q\npIe1PjmkOYcHFyWk+wYrhXSbzfh0o8+Ta+pumyi3ihrS98NS7/+eWMZ8SLcnqsXtx09Vu+Ps\nDkfLU/NiSIP1Pd75uv5L2/xZm72exUOqOHZLSF3/RHCdkMo+98XNhtRpZ7Y6pD/dt4V+hDS+\n7e9qk+tr3V7uU9ihG+3QsO7hrObOVxfS7DbLhqQcszWky2idkJRzUveI1OmbVIZ0WPzFhv51\nk2P3eMHE+ua2P463v94aDQeoX/doNrjWqftWTUjH+W0WDUl1ODaFdLunrhZShU9FSMfZTfKE\nNBipIZW2P463H17FQiF18k/2kPqzRkhuk7pzooU02LRl+U/7gWxrSJbwnhPS9LVO3flqQuKp\nXZhJXUBqJSFP7ZajuIyZkLqjsv1ApztqOwSE1E2vb3TnO9aHVLqeZ4SkHbsxpNtfnhuSfk7u\nr+TXhdR+RkYh/e/759O6bwu8+bv4c+HCBa3bH292DTu0rFv8Vf259+BH5NPbdv22xW3Mq65R\n0Y/54KRveH95Z3Glx2uoPSf9C1BlResZGf7jJ1/P3x8dDu2/lALghRmE9OPwfvqh7L/8h30A\nLUy8s+H2fwCohZAAAph+avf+nH+OC2AvDF9suP2bDfy35gANjJ7C/Tz/mw3P/aeLAbYO3wsB\nBEBIAAE87U2rAHuGkAACmAzmz7efz14HwKaZfuT5e6AkgAYKT+HsT+1+JcPqkU8Fk8xMB/Ov\n7Rcxn1j7JA0x3zLpVDDJTOnFhnfrFa59koY4bpu1lz4Ak8xMh9SZO8p2zrj77cskLeEvc699\nkobsRwWTzBDSZlQwyUzxB7LWH8qufZKGOG6btZc+AJPMENJmVDDJzOj3I53+AaE/3+z/cPHa\nJ2mI47ZZe+kDMMlM4R/Rt//jJ2ufpCGO22btpQ/AJDPTv9blL+9syKeCSWYGwXw7nJ7U/fl2\n+G69wrVP0hDHbbP20gdgkplBSL/d/2ZDzQ35duH6l/sH8Vn8pb/o7U3u+LDpIiftKSZi/1VN\nglSWNknL8Cnc3/fTv9ngeO931Tl7OFvipA1O2eWS60WDLSrP2dIhuU0qNRY38au8jTZewCQt\nq/xAduKcnT++DbaaCemt9h64rIrfJHNIbSqEFEr8ObteNLz7bTCkCZP6jnKF5FJxmKQl/LdR\nNJ6z/lQNH2P6S7cRks2k/lukJ4ZkVPnF90h3/L+Nouqc9d/Xiq95g5Mwf86yhBRhUqmyrEmA\nSvVZcZikJfy3UVSds8Gfpp4a3Gdv1xM2vOPlCMlpMryW9Uz8KvVnxWGSlvB/RL/xnBVfz+5f\nVJ0I6a3+GZHjtnmCyfBa1jPxqxCS/OuzQxp9HF1Suvvt4xEp0VO756k4TNIS/tsoms7Zm/z8\nNrXJ7cl49pCsJtXfVzwvJI8KIV3x/zaKp5yz6lOWPqT6twOkD4l3Njzg/W0UVbfkE3HcNmsv\nfQAmmeE/Nd+MCiaZGb772/2b+py38VvLK3I1rKayH5NwFYdJWgYhde5HqKgbO4r9qGCSmeF/\nRvHt3flLL9c+SUP2o4JJZsJ/rcvaJ2mI47ZZe+kDMMkMIW1GBZPM8KrdZlQwyUx4SB83+j99\nzM6qN7RdYYDKcebqlb9G7rrUSSmPl5o7TNIiQwr5fZezNyohOXZd6qSUx4RUzygkb02zNyoh\nOXZd6qSUx4RUDyHpKoQUPHeYpIWQdBVCCp47TNJCSLoKIQXPHSZpISRdhZCC5w6TtBCSrkJI\nwXOHSVoeQ3L+bqQTszcqITl2DTAhpMUgJF2FkILnDpO08M4GXYWQgucOk7QQkq5CSMFzh0la\nCElXIaTgucMkLYSkqxBS8NxhkhZC0lUIKXjuMEnLUiF9+aTyhk4e0pehymZDGpmUjrX03GGS\nFkLSVAgpfO4wSQshaSqEFD53mKSFkDQVQgqfO0zSQkiaCiGFzx0maSEkTYWQwucOk7QQkqZC\nSOFzh0laCElTIaTwucMkLYSkqRBS+NxhkhZC0lQIKXzuMEkLIWkqhBQ+d5ikhZA0FUIKnztM\n0kJImgohhc8dJmkhJI3z3S/ghlmT/pwQ0kIQkqbCI1L43GGSFkLSVAgpfO4wSQshaSqEFD53\nmKSFkDQVQgqfO0zSQkiaCiGFzx0maSEkTYWQwucOk7QQkqZCSOFzh0laCElT2UNIF/bwE7G0\nEJKmsoeQCialYy09d5ikhZA0FUIKnztM0kJImgohhc8dJmkhJE2FkMLnDpO0EJKmQkjhc4dJ\nWghJUyGk8LnDJC2EpKkQUvjcYZKW8JCu7OFHFoW7HyER0hgekTQVQgqfO0zSQkiaCiGFzx0m\naSEkTYWQwucOk7QQkqZCSOFzh0laCElTIaTwucMkLYSkqRBS+NxhkhZC0lQIKXzuMEkLIWkq\nhBQ+d5ikhZA0FUIKnztM0kJImgohhc8dJmkhJE2FkMLnDpO0EJKmQkjhc4dJWghJUyGk8LnD\nJC2EpKkQUvjcYZIWQtJUCCl87jBJCyFpKoQUPneYpIWQNBVCCp87TNJCSJoKIYXPHSZpISRN\nhZDC5w6TtBCSpkJI4XOHSVoISVMhpPC5wyQthKSpEFL43GGSFkLSVAgpfO4wSQshaSqEFD53\nmKSFkDQVQgqfO0zSQkiaCiGFzx0maSEkTYWQwucOk7QQkqZCSOFzh0laCElTIaTwucMkLYSk\nqRBS+NxhkhZC0lQIKXzuMEkLIWkqhBQ+d5ikhZA0FUIKnztM0kJImgohhc8dJmkhJE2FkMLn\nDpO0EJKmQkjhc4dJWl4qpO6Tms8PKoQUPneYpOWVQuquH7TPjyqEFD53mKSFkAiJkAJ4pZDO\nENIEhOSGkCZD+ufEZYfz3S/6Vnoy/TkhpIV4tZC6I49IYwjJDSFtM6S2FyALJqVjLT0Pu7Ml\n4sVC6uSHDYfU+LpJwaR0rKXnYXe2RLxWSF3/kZDKx1p6HnZnS8RLhdSJT9sO6agoENKzeaWQ\nuu76ncNO3tlQFdLOXn9MyyuFZFNJG1LD6yYFk9Kxlp47zklaKkKyvEBESO3bEtKW0UMyfV9L\nSO3bNobU8gJkwaR0rKXnjnOSFkIq05skDKnpBciCSelYS88d5yQtld8jEVLxmOuE1PYCZMGk\ndKyl545zkpbQkPoXiI57eIGocPfLEFLjC5AFk9Kxlp4/7Qw+kbqQmr+v5RGpfdvWFxv8JqVj\nLT13mKSFkMr0JoQUOneYpKUqpPYXiAipfVtC2jI1IRleICKk9m0JacvU/EBWfCIkQvLPHSZp\nqfg5kuUFIkJq35aQtgzvtdNUCCl87jBJCyFpKoQUPneYpIWQNBVCCp87TNJCSJoKIYXPHSZp\nISRNhZDC5w6TtBCSpkJI4XOHSVoISVMhpPC5wyQthKSpEFL43GGSFkLSVAgpfO4wSQshaSqE\nFD53mKSFkDQVQgqfO0zSQkiaCiGFzx0maSEkTYWQwucOk7QQkqZCSOFzh0laCElTIaTwucMk\nLYSkqRBS+NxhkhZC0lQIKXzuMEkLIWkqhBQ+d5ikhZA0FUIKnztM0kJImgohhc8dJmkhJE2F\nkMLnDpO0EJKmQkjhc4dJWghJUyGk8LnDJC2EpKkQUvjcYZIWQtJUCCl87jBJCyFpKoQUPneY\npIWQNBVCCp87TNJCSJoKIYXPHSZpISRNhZDC5w6TtBCSpkJI4XOHSVoISVMhpPC5wyQthKSp\nEFL43GGSFkLSVAgpfO4wSQshaSqEFD53mKSFkDQVQgqfO0zSQkiaCiGFzx0maSEkTYWQwucO\nk7QQkqZCSOFzh0laCEnjfPcLuGHWpD8nhLQQhKSp8IgUPneYpIWQNBVCCp87TNJCSJoKIYXP\nHSZpISRNZQ8hXdjDd3tpISRNZQ8hFUxKx1p67jBJCyFpKoQUPneYpIWQNBVCCp87TNJCSJoK\nIYXPHSZpCQ/pyh6+ry3c/QiJkMbwiKSpEFL43GGSFkLSVAgpfO4wSQshaSqEFD53mKSFkDQV\nQgqfO0zSQkiaCiGFzx0maSEkTYWQwucOk7QQkqZCSOFzh0laCElTIaTwucMkLYSkqRBS+Nxh\nkhZC0lQIKXzuMEkLIWkqhBQ+d5ikhZA0FUIKnztM0kJImgohhc8dJmkhJE2FkMLnDpO0EJKm\nQkjhc4dJWghJUyGk8LnDJC2EpKkQUvjcYZIWQtJUCCl87jBJCyFpKoQUPneYpIWQNBVCCp87\nTNJCSJoKIYXPHSZpISRNhZDC5w6TtBCSpkJI4XOHSVoISVMhpPC5wyQthKSpEFL43GGSFkLS\nVAgpfO4wSQshaSqEFD53mKSFkDQVQgqfO0zSQkiaCiGFzx0maSEkTYWQwucOk7QQkqZCSOFz\nh0laCElTIaTwucMkLYSkqRBS+NxhkhZC0lQIKXzuMEkLIWkqhBQ+d5ikhZA0FUIKnztM0kJI\nmgohhc8dJmkhJE2FkMLnDpO0EJKmQkjhc4dJWghJUyGk8LnDJC2EpKkQUvjcYZIWQtJUCCl8\n7jBJCyFpKoQUPneYpOXFQuouHz+Z+/ygQkjhc4dJWl4rpGsv1w+lz48qhBQ+d5ik5aVC6o6E\n9DEFIbl5qZCOewqp5VlqwaR0rKXnMfe0XBDSZEj/nLjscr77Rd9KfpqepfbnhJAWgpC2+YjU\n9iy1YFI61tLzqPtaJqpCan8WQUjt21qe2hFSFmpCMjyLIKT2bZcJKf+T1H1QEZLlWQQhtW/L\nI9KWqX9qR0jTxySk5nnNfW5rhIbUP4s4Jn0W0fLdHiERUj0v9ojURG9CSKFzxzlJCyGV6U0I\nKXTuOCdpIaQyvdRTMsQAAAm+SURBVEnakHhnQxYIqUxvkjGkAJPSsZaeO0zSQkhlehNCCp07\nTNLyYu9saKI3IaTQucMkLS/2XjuDCiGFzx0maSEkTYWQwucOk7QQkqZCSOFzh0laCElTIaTw\nucMkLYSkqRBS+NxhkhZC0lQIKXzuMEkLIWkqhBQ+d5ikhZA0FUIKnztM0kJImgohhc8dJmkh\nJE2FkMLnDpO0EJKmQkjhc4dJWghJUyGk8LnDJC2EpKkQUvjcYZIWQtJUCCl87jBJCyFpKoQU\nPneYpIWQNBVCCp87TNJCSJoKIYXPHSZpISRNhZDC5w6TtBCSpkJI4XOHSVoISVMhpPC5wyQt\nhKSpEFL43GGSFkLSVAgpfO4wSQshaSqEFD53mKSFkDQVQgqfO0zSQkiaCiGFzx0maSEkTYWQ\nwucOk7QQkqZCSOFzh0laCElTIaTwucMkLYSkqRBS+NxhkhZC0lQIKXzuMEkLIWkqhBQ+d5ik\nhZA0FUIKnztM0kJImgohhc8dJmkhJE2FkMLnDpO0EJKmQkjhc4dJWghJUyGk8LnDJC2EpKkQ\nUvjcYZIWQtJUCCl87jBJCyFpKoQUPneYpIWQNBVCCp87TNJCSJoKIYXPHSZpISRNhZDC5w6T\ntBCSpkJI4XOHSVoISeN89wu4YdakPyeEtBCEpKnwiBQ+d5ikhZA0lT2EdGEPj61pISRNZQ8h\nFUxKx1p67jBJS3hIV/bwxa9w9yMkQhrDI5KmQkjhc4dJWghJUyGk8LnDJC2EpKkQUvjcYZIW\nQtJUCCl87jBJCyFpKoQUPneYpIWQNBVCCp87TNJCSJoKIYXPHSZpISRNhZDC5w6TtBCSpkJI\n4XOHSVoISVMhpPC5wyQthKSpEFL43GGSFkLSVAgpfO4wSQshaSqEFD53mKSFkDQVQgqfO0zS\nQkiaCiGFzx0maSEkTYWQwucOk7QQkqZCSOFzh0laCElTIaTwucMkLYSkqRBS+NxhkhZC0lQI\nKXzuMEkLIWkqX24UjklIzXOHSVoISVMhpPC5wyQthKSpEFL43GGSFkLSVAgpfO4wSQshaSqE\nFD53mKSFkDQVQgqfO0zSQkiaCiGFzx0maSEkTYWQwucOk7QQkqays5AmTwwhuSEkTYWQwucO\nk7QQkqZCSOFzh0laCElTIaTwucMkLYSkqRBS+NxhkhZC0lQIKXzuMEkLIWkqhBQ+d5ikZcmQ\nxieNkBy7xpuUjrX03GGSFkLSVAgpfO4wSQshaSqEFD53mKSFkDQVQgqfO0zSQkiaCiGFzx0m\naSEkTYWQwucOk7QQkqZCSOFzh0laCElTIaTwucMkLYSkqRBS+NxhkhZC0lQIKXzuMEkLIWkq\nhBQ+d5ikhZA0FUIKnztM0kJImgohhc8dJmkJCKn7pP/bx9xJSx7So8nw7nez2URIUydlZFI6\n1tLzJpON4A+pu3848zF30nKHNDDZckiTJ4WQFoSQiiZTd7+JZ0YbDWnu2WppDVHzFpOt8JSQ\n7uwjpMeYthzS7HkZryFq3mKyFUJD+ucT9/Wtx8BkHypbN9kKSz0iWR9AAmbBj0ixDyuOXZc6\nKeUxj0j1EFLRhJAIqR5CKpoQEiHVQ0hFE0IipHoIqWhCSIRUz1LvbNhiSNPvbNhkSC0npTwm\npHqWeq/dJkMqqGwxpIaTUh4TUj2EpKsQUvDcYZIWQtJVCCl47jBJCyHpKoQUPHeYpIWQdBVC\nCp47TNJCSLoKIQXPHSZpISRdhZCC5w6TtBCSrkJIwXOHSVrCQ7pT/d792g3DrzDm+mYvXWxX\nI41X2rqGF/7vNQjJe32EZN9+RxCS9/oIyb79jiAk7/URkn37HUFI3usjJPv2O2K5kABeCEIC\nCICQAAIgJIAACAkggKCQBv+R9viC4efZDS9b1G3XcIVVaNdXuLpOMrlr6cKHK57etd2iSmWw\ngeGWDL3lt01MSMN/NmR0wfBz+RruF06fg+F2DVdYhXZ93fQG3W3B80uZveJAi8d9Sxc8bmC4\nJetO0WuwWEjdwwUzIT1ueL+wG56H6e3qrzDEpBtt0F86EdJwKbNXHGjxuG/pStWQlDVUnqLX\nIDik/sF85pypG8p99O2uf1A2jDUpXjq/a9UVB1i0qIz2qV3DEmveLrEhiVMzH1J5w9t3DOKL\n2vx2XVe1YbzJ7NGKFzp2rbdoVZH7VK+h9hS9BsuEdP++euqcyVu/tGFXud1RVqceOdakdOl9\nSYMLH4t/uLRTdm21aFWRu9RuXn2KXoPgkPqvQuUvfpd7TGHDy1+6yu0qjnyUn8NM5i6dWUrh\niruKXZssWlSGu1Suof4UvQYLPCKJ0dQ5G56+yZAevrqVt6s48lF+jjUZX3qtrLiUwhX3uuVd\nmyxsKlWn5n5Z9Sl6DZ4ekvzaNbvh5cudtl37vSXMZHzp7dWE4q7aFc/s2mThVqlcQ9Upeg0W\ne7GhcEH3OJ/cUGyjbld15IazWW8yf7TpC6d+DlW5a/t9skHlKP7UsobwNW+X4Hc23J8uFy64\nP6HWNjzev8or29UeOcrkIjB5qWS86+2C2SuOsqhSebyg5dTcL6m/+r3De+0AAiAkgAAICSAA\nQgIIgJAAAiAkgAAICSAAQgIIgJAAAiAkgAAICSAAQtoDB07j2nAG9gAhrQ5nYA8Q0upwBjbG\n38PX8+evh9/HX98Ph+79eAnpEtP5498fh8OPvysu8gUhpK3x/fDn8+Ofz57+O5x5H4XUncZf\n113mq0FIW+O/UznH98N/nw9K/x6Pv28R9SH9PG3xfvjfygt9LQhpc3y9/Gespz/++e/nt3FI\nXy9/+r7iGl8PQtoc/zv8Ov46/Pz807fLc7thSIfDbQ5Pg1t7c/w9/Ph84vb3ePxx+Pq///4Q\nUgq4tbfHj8Of8/O2yyt0DyH96Z/awVPhNt8evz4fbX4dT+n8Ov69f4/UHf69/u399GLDv4dv\na6/zpSCkDfL18tr2+0F+j3T+28/Tn/6eX/4+/F57mS8FIW2Q/51e9z6enuMdvv26P6177w4/\nr0/wzhesusSXg5AAAiAkgAAICSAAQgIIgJAAAiAkgAAICSAAQgIIgJAAAiAkgAAICSAAQgII\ngJAAAiAkAEnlf1483IaQACSHh0+zWx2GAwC4QkgAAYiQbk/fTp/k3webjv4MAH1IB/H5cFAe\nhQgJQDJ4sWEQlAyGFxsAijw8zBzkI9Hw5TxCAigyKGX4iDS54ehvAK/O4fFPxZBmswJ4dR5D\nuj21G73YMAyHkAAk8qld/7L38OXv0dsfCAmgAi0UQgKYZ/Tzo+JGAFCm6k2sz1gIwN4hJIAA\nCAkgAEICCICQAAIgJIAACAkgAEICCICQAAL4PwB76zqmHBq1AAAAAElFTkSuQmCC",
      "text/plain": [
       "plot without title"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_histogram(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can make a few observations from the above histogram. The distribution above shows that all nearly all PAY attributes are rightly skewed."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4) Feature Engineering\n",
    "\n",
    "This step can be more important than the actual model used because a machine learning algorithm only learns from the data we give it, and creating features that are relevant to a task is absolutely crucial.\n",
    "\n",
    "Analyzing our data above, we've been able to note the extremely week correlation of some variables with the final target variable. The following are the ones which have significantly low correlation values: AGE, BILL_AMT2, BILL_AMT3, BILL_AMT4, BILL_AMT5, BILL_AMT6. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<thead><tr><th></th><th scope=col>LIMIT_BAL</th><th scope=col>SEX</th><th scope=col>EDUCATION</th><th scope=col>MARRIAGE</th><th scope=col>PAY_0</th><th scope=col>PAY_2</th><th scope=col>PAY_3</th><th scope=col>PAY_4</th><th scope=col>PAY_5</th><th scope=col>PAY_6</th><th scope=col>BILL_AMT1</th><th scope=col>PAY_AMT1</th><th scope=col>PAY_AMT2</th><th scope=col>PAY_AMT3</th><th scope=col>PAY_AMT4</th><th scope=col>PAY_AMT5</th><th scope=col>PAY_AMT6</th><th scope=col>default_payment</th></tr></thead>\n",
       "<tbody>\n",
       "\t<tr><th scope=row>2</th><td> 20000</td><td>2     </td><td>2     </td><td>1     </td><td> 2    </td><td>2     </td><td>-1    </td><td>-1    </td><td>-2    </td><td>-2    </td><td> 3913 </td><td>   0  </td><td>  689 </td><td>    0 </td><td>   0  </td><td>   0  </td><td>   0  </td><td>1     </td></tr>\n",
       "\t<tr><th scope=row>3</th><td>120000</td><td>2     </td><td>2     </td><td>2     </td><td>-1    </td><td>2     </td><td> 0    </td><td> 0    </td><td> 0    </td><td> 2    </td><td> 2682 </td><td>   0  </td><td> 1000 </td><td> 1000 </td><td>1000  </td><td>   0  </td><td>2000  </td><td>1     </td></tr>\n",
       "\t<tr><th scope=row>4</th><td> 90000</td><td>2     </td><td>2     </td><td>2     </td><td> 0    </td><td>0     </td><td> 0    </td><td> 0    </td><td> 0    </td><td> 0    </td><td>29239 </td><td>1518  </td><td> 1500 </td><td> 1000 </td><td>1000  </td><td>1000  </td><td>5000  </td><td>0     </td></tr>\n",
       "\t<tr><th scope=row>5</th><td> 50000</td><td>2     </td><td>2     </td><td>1     </td><td> 0    </td><td>0     </td><td> 0    </td><td> 0    </td><td> 0    </td><td> 0    </td><td>46990 </td><td>2000  </td><td> 2019 </td><td> 1200 </td><td>1100  </td><td>1069  </td><td>1000  </td><td>0     </td></tr>\n",
       "\t<tr><th scope=row>6</th><td> 50000</td><td>1     </td><td>2     </td><td>1     </td><td>-1    </td><td>0     </td><td>-1    </td><td> 0    </td><td> 0    </td><td> 0    </td><td> 8617 </td><td>2000  </td><td>36681 </td><td>10000 </td><td>9000  </td><td> 689  </td><td> 679  </td><td>0     </td></tr>\n",
       "\t<tr><th scope=row>7</th><td> 50000</td><td>1     </td><td>1     </td><td>2     </td><td> 0    </td><td>0     </td><td> 0    </td><td> 0    </td><td> 0    </td><td> 0    </td><td>64400 </td><td>2500  </td><td> 1815 </td><td>  657 </td><td>1000  </td><td>1000  </td><td> 800  </td><td>0     </td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "\\begin{tabular}{r|llllllllllllllllll}\n",
       "  & LIMIT\\_BAL & SEX & EDUCATION & MARRIAGE & PAY\\_0 & PAY\\_2 & PAY\\_3 & PAY\\_4 & PAY\\_5 & PAY\\_6 & BILL\\_AMT1 & PAY\\_AMT1 & PAY\\_AMT2 & PAY\\_AMT3 & PAY\\_AMT4 & PAY\\_AMT5 & PAY\\_AMT6 & default\\_payment\\\\\n",
       "\\hline\n",
       "\t2 &  20000 & 2      & 2      & 1      &  2     & 2      & -1     & -1     & -2     & -2     &  3913  &    0   &   689  &     0  &    0   &    0   &    0   & 1     \\\\\n",
       "\t3 & 120000 & 2      & 2      & 2      & -1     & 2      &  0     &  0     &  0     &  2     &  2682  &    0   &  1000  &  1000  & 1000   &    0   & 2000   & 1     \\\\\n",
       "\t4 &  90000 & 2      & 2      & 2      &  0     & 0      &  0     &  0     &  0     &  0     & 29239  & 1518   &  1500  &  1000  & 1000   & 1000   & 5000   & 0     \\\\\n",
       "\t5 &  50000 & 2      & 2      & 1      &  0     & 0      &  0     &  0     &  0     &  0     & 46990  & 2000   &  2019  &  1200  & 1100   & 1069   & 1000   & 0     \\\\\n",
       "\t6 &  50000 & 1      & 2      & 1      & -1     & 0      & -1     &  0     &  0     &  0     &  8617  & 2000   & 36681  & 10000  & 9000   &  689   &  679   & 0     \\\\\n",
       "\t7 &  50000 & 1      & 1      & 2      &  0     & 0      &  0     &  0     &  0     &  0     & 64400  & 2500   &  1815  &   657  & 1000   & 1000   &  800   & 0     \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "| <!--/--> | LIMIT_BAL | SEX | EDUCATION | MARRIAGE | PAY_0 | PAY_2 | PAY_3 | PAY_4 | PAY_5 | PAY_6 | BILL_AMT1 | PAY_AMT1 | PAY_AMT2 | PAY_AMT3 | PAY_AMT4 | PAY_AMT5 | PAY_AMT6 | default_payment |\n",
       "|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|\n",
       "| 2 |  20000 | 2      | 2      | 1      |  2     | 2      | -1     | -1     | -2     | -2     |  3913  |    0   |   689  |     0  |    0   |    0   |    0   | 1      |\n",
       "| 3 | 120000 | 2      | 2      | 2      | -1     | 2      |  0     |  0     |  0     |  2     |  2682  |    0   |  1000  |  1000  | 1000   |    0   | 2000   | 1      |\n",
       "| 4 |  90000 | 2      | 2      | 2      |  0     | 0      |  0     |  0     |  0     |  0     | 29239  | 1518   |  1500  |  1000  | 1000   | 1000   | 5000   | 0      |\n",
       "| 5 |  50000 | 2      | 2      | 1      |  0     | 0      |  0     |  0     |  0     |  0     | 46990  | 2000   |  2019  |  1200  | 1100   | 1069   | 1000   | 0      |\n",
       "| 6 |  50000 | 1      | 2      | 1      | -1     | 0      | -1     |  0     |  0     |  0     |  8617  | 2000   | 36681  | 10000  | 9000   |  689   |  679   | 0      |\n",
       "| 7 |  50000 | 1      | 1      | 2      |  0     | 0      |  0     |  0     |  0     |  0     | 64400  | 2500   |  1815  |   657  | 1000   | 1000   |  800   | 0      |\n",
       "\n"
      ],
      "text/plain": [
       "  LIMIT_BAL SEX EDUCATION MARRIAGE PAY_0 PAY_2 PAY_3 PAY_4 PAY_5 PAY_6\n",
       "2  20000    2   2         1         2    2     -1    -1    -2    -2   \n",
       "3 120000    2   2         2        -1    2      0     0     0     2   \n",
       "4  90000    2   2         2         0    0      0     0     0     0   \n",
       "5  50000    2   2         1         0    0      0     0     0     0   \n",
       "6  50000    1   2         1        -1    0     -1     0     0     0   \n",
       "7  50000    1   1         2         0    0      0     0     0     0   \n",
       "  BILL_AMT1 PAY_AMT1 PAY_AMT2 PAY_AMT3 PAY_AMT4 PAY_AMT5 PAY_AMT6\n",
       "2  3913        0       689        0       0        0        0    \n",
       "3  2682        0      1000     1000    1000        0     2000    \n",
       "4 29239     1518      1500     1000    1000     1000     5000    \n",
       "5 46990     2000      2019     1200    1100     1069     1000    \n",
       "6  8617     2000     36681    10000    9000      689      679    \n",
       "7 64400     2500      1815      657    1000     1000      800    \n",
       "  default_payment\n",
       "2 1              \n",
       "3 1              \n",
       "4 0              \n",
       "5 0              \n",
       "6 0              \n",
       "7 0              "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#deleting columns\n",
    "\n",
    "data_new <- select(data, -one_of('ID','AGE', 'BILL_AMT2',\n",
    "       'BILL_AMT3','BILL_AMT4','BILL_AMT5','BILL_AMT6'))\n",
    "\n",
    "head(data_new)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5) Pre-processing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Standardization is a transformation that centers the data by removing the mean value of each feature and then scale it by dividing (non-constant) features by their standard deviation. After standardizing data the mean will be zero and the standard deviation one. It is most suitable for techniques that assume a Gaussian distribution in the input variables and work better with rescaled data, such as linear regression, logistic regression and linear discriminate analysis. If a feature has a variance that is orders of magnitude larger than others, it might dominate the objective function and make the estimator unable to learn from other features correctly as expected.\n",
    "\n",
    "In the code below, we'll use the scale method transform our dataset using it. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_new[, 1:17] <- scale(data_new[, 1:17])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<thead><tr><th></th><th scope=col>LIMIT_BAL</th><th scope=col>SEX</th><th scope=col>EDUCATION</th><th scope=col>MARRIAGE</th><th scope=col>PAY_0</th><th scope=col>PAY_2</th><th scope=col>PAY_3</th><th scope=col>PAY_4</th><th scope=col>PAY_5</th><th scope=col>PAY_6</th><th scope=col>BILL_AMT1</th><th scope=col>PAY_AMT1</th><th scope=col>PAY_AMT2</th><th scope=col>PAY_AMT3</th><th scope=col>PAY_AMT4</th><th scope=col>PAY_AMT5</th><th scope=col>PAY_AMT6</th><th scope=col>default_payment</th></tr></thead>\n",
       "<tbody>\n",
       "\t<tr><th scope=row>2</th><td>-1.1367012 </td><td> 0.8101472 </td><td> 0.2118664 </td><td>-1.0687794 </td><td> 1.79453395</td><td>1.7823185  </td><td>-0.6966518 </td><td>-0.6665876 </td><td>-1.5300205 </td><td>-1.4860160 </td><td>-0.64249036</td><td>-0.3419359 </td><td>-0.2270819 </td><td>-0.2967963 </td><td>-0.3080574 </td><td>-0.3141309 </td><td>-0.29337717</td><td>1          </td></tr>\n",
       "\t<tr><th scope=row>3</th><td>-0.3659744 </td><td> 0.8101472 </td><td> 0.2118664 </td><td> 0.8491164 </td><td>-0.87497656</td><td>1.7823185  </td><td> 0.1388625 </td><td> 0.1887429 </td><td> 0.2349126 </td><td> 1.9922823 </td><td>-0.65920776</td><td>-0.3419359 </td><td>-0.2135841 </td><td>-0.2400006 </td><td>-0.2442256 </td><td>-0.3141309 </td><td>-0.18087519</td><td>1          </td></tr>\n",
       "\t<tr><th scope=row>4</th><td>-0.5971924 </td><td> 0.8101472 </td><td> 0.2118664 </td><td> 0.8491164 </td><td> 0.01486028</td><td>0.1117342  </td><td> 0.1388625 </td><td> 0.1887429 </td><td> 0.2349126 </td><td> 0.2531332 </td><td>-0.29855468</td><td>-0.2502874 </td><td>-0.1918835 </td><td>-0.2400006 </td><td>-0.2442256 </td><td>-0.2486786 </td><td>-0.01212223</td><td>0          </td></tr>\n",
       "\t<tr><th scope=row>5</th><td>-0.9054832 </td><td> 0.8101472 </td><td> 0.2118664 </td><td>-1.0687794 </td><td> 0.01486028</td><td>0.1117342  </td><td> 0.1388625 </td><td> 0.1887429 </td><td> 0.2349126 </td><td> 0.2531332 </td><td>-0.05749007</td><td>-0.2211869 </td><td>-0.1693583 </td><td>-0.2286415 </td><td>-0.2378424 </td><td>-0.2441624 </td><td>-0.23712618</td><td>0          </td></tr>\n",
       "\t<tr><th scope=row>6</th><td>-0.9054832 </td><td>-1.2343024 </td><td> 0.2118664 </td><td>-1.0687794 </td><td>-0.87497656</td><td>0.1117342  </td><td>-0.6966518 </td><td> 0.1887429 </td><td> 0.2349126 </td><td> 0.2531332 </td><td>-0.57860845</td><td>-0.2211869 </td><td> 1.3350119 </td><td> 0.2711608 </td><td> 0.2664292 </td><td>-0.2690343 </td><td>-0.25518275</td><td>0          </td></tr>\n",
       "\t<tr><th scope=row>7</th><td>-0.9054832 </td><td>-1.2343024 </td><td>-1.1313270 </td><td> 0.8491164 </td><td> 0.01486028</td><td>0.1117342  </td><td> 0.1388625 </td><td> 0.1887429 </td><td> 0.2349126 </td><td> 0.2531332 </td><td> 0.17894364</td><td>-0.1909996 </td><td>-0.1782122 </td><td>-0.2594815 </td><td>-0.2442256 </td><td>-0.2486786 </td><td>-0.24837638</td><td>0          </td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "\\begin{tabular}{r|llllllllllllllllll}\n",
       "  & LIMIT\\_BAL & SEX & EDUCATION & MARRIAGE & PAY\\_0 & PAY\\_2 & PAY\\_3 & PAY\\_4 & PAY\\_5 & PAY\\_6 & BILL\\_AMT1 & PAY\\_AMT1 & PAY\\_AMT2 & PAY\\_AMT3 & PAY\\_AMT4 & PAY\\_AMT5 & PAY\\_AMT6 & default\\_payment\\\\\n",
       "\\hline\n",
       "\t2 & -1.1367012  &  0.8101472  &  0.2118664  & -1.0687794  &  1.79453395 & 1.7823185   & -0.6966518  & -0.6665876  & -1.5300205  & -1.4860160  & -0.64249036 & -0.3419359  & -0.2270819  & -0.2967963  & -0.3080574  & -0.3141309  & -0.29337717 & 1          \\\\\n",
       "\t3 & -0.3659744  &  0.8101472  &  0.2118664  &  0.8491164  & -0.87497656 & 1.7823185   &  0.1388625  &  0.1887429  &  0.2349126  &  1.9922823  & -0.65920776 & -0.3419359  & -0.2135841  & -0.2400006  & -0.2442256  & -0.3141309  & -0.18087519 & 1          \\\\\n",
       "\t4 & -0.5971924  &  0.8101472  &  0.2118664  &  0.8491164  &  0.01486028 & 0.1117342   &  0.1388625  &  0.1887429  &  0.2349126  &  0.2531332  & -0.29855468 & -0.2502874  & -0.1918835  & -0.2400006  & -0.2442256  & -0.2486786  & -0.01212223 & 0          \\\\\n",
       "\t5 & -0.9054832  &  0.8101472  &  0.2118664  & -1.0687794  &  0.01486028 & 0.1117342   &  0.1388625  &  0.1887429  &  0.2349126  &  0.2531332  & -0.05749007 & -0.2211869  & -0.1693583  & -0.2286415  & -0.2378424  & -0.2441624  & -0.23712618 & 0          \\\\\n",
       "\t6 & -0.9054832  & -1.2343024  &  0.2118664  & -1.0687794  & -0.87497656 & 0.1117342   & -0.6966518  &  0.1887429  &  0.2349126  &  0.2531332  & -0.57860845 & -0.2211869  &  1.3350119  &  0.2711608  &  0.2664292  & -0.2690343  & -0.25518275 & 0          \\\\\n",
       "\t7 & -0.9054832  & -1.2343024  & -1.1313270  &  0.8491164  &  0.01486028 & 0.1117342   &  0.1388625  &  0.1887429  &  0.2349126  &  0.2531332  &  0.17894364 & -0.1909996  & -0.1782122  & -0.2594815  & -0.2442256  & -0.2486786  & -0.24837638 & 0          \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "| <!--/--> | LIMIT_BAL | SEX | EDUCATION | MARRIAGE | PAY_0 | PAY_2 | PAY_3 | PAY_4 | PAY_5 | PAY_6 | BILL_AMT1 | PAY_AMT1 | PAY_AMT2 | PAY_AMT3 | PAY_AMT4 | PAY_AMT5 | PAY_AMT6 | default_payment |\n",
       "|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|\n",
       "| 2 | -1.1367012  |  0.8101472  |  0.2118664  | -1.0687794  |  1.79453395 | 1.7823185   | -0.6966518  | -0.6665876  | -1.5300205  | -1.4860160  | -0.64249036 | -0.3419359  | -0.2270819  | -0.2967963  | -0.3080574  | -0.3141309  | -0.29337717 | 1           |\n",
       "| 3 | -0.3659744  |  0.8101472  |  0.2118664  |  0.8491164  | -0.87497656 | 1.7823185   |  0.1388625  |  0.1887429  |  0.2349126  |  1.9922823  | -0.65920776 | -0.3419359  | -0.2135841  | -0.2400006  | -0.2442256  | -0.3141309  | -0.18087519 | 1           |\n",
       "| 4 | -0.5971924  |  0.8101472  |  0.2118664  |  0.8491164  |  0.01486028 | 0.1117342   |  0.1388625  |  0.1887429  |  0.2349126  |  0.2531332  | -0.29855468 | -0.2502874  | -0.1918835  | -0.2400006  | -0.2442256  | -0.2486786  | -0.01212223 | 0           |\n",
       "| 5 | -0.9054832  |  0.8101472  |  0.2118664  | -1.0687794  |  0.01486028 | 0.1117342   |  0.1388625  |  0.1887429  |  0.2349126  |  0.2531332  | -0.05749007 | -0.2211869  | -0.1693583  | -0.2286415  | -0.2378424  | -0.2441624  | -0.23712618 | 0           |\n",
       "| 6 | -0.9054832  | -1.2343024  |  0.2118664  | -1.0687794  | -0.87497656 | 0.1117342   | -0.6966518  |  0.1887429  |  0.2349126  |  0.2531332  | -0.57860845 | -0.2211869  |  1.3350119  |  0.2711608  |  0.2664292  | -0.2690343  | -0.25518275 | 0           |\n",
       "| 7 | -0.9054832  | -1.2343024  | -1.1313270  |  0.8491164  |  0.01486028 | 0.1117342   |  0.1388625  |  0.1887429  |  0.2349126  |  0.2531332  |  0.17894364 | -0.1909996  | -0.1782122  | -0.2594815  | -0.2442256  | -0.2486786  | -0.24837638 | 0           |\n",
       "\n"
      ],
      "text/plain": [
       "  LIMIT_BAL  SEX        EDUCATION  MARRIAGE   PAY_0       PAY_2     PAY_3     \n",
       "2 -1.1367012  0.8101472  0.2118664 -1.0687794  1.79453395 1.7823185 -0.6966518\n",
       "3 -0.3659744  0.8101472  0.2118664  0.8491164 -0.87497656 1.7823185  0.1388625\n",
       "4 -0.5971924  0.8101472  0.2118664  0.8491164  0.01486028 0.1117342  0.1388625\n",
       "5 -0.9054832  0.8101472  0.2118664 -1.0687794  0.01486028 0.1117342  0.1388625\n",
       "6 -0.9054832 -1.2343024  0.2118664 -1.0687794 -0.87497656 0.1117342 -0.6966518\n",
       "7 -0.9054832 -1.2343024 -1.1313270  0.8491164  0.01486028 0.1117342  0.1388625\n",
       "  PAY_4      PAY_5      PAY_6      BILL_AMT1   PAY_AMT1   PAY_AMT2   PAY_AMT3  \n",
       "2 -0.6665876 -1.5300205 -1.4860160 -0.64249036 -0.3419359 -0.2270819 -0.2967963\n",
       "3  0.1887429  0.2349126  1.9922823 -0.65920776 -0.3419359 -0.2135841 -0.2400006\n",
       "4  0.1887429  0.2349126  0.2531332 -0.29855468 -0.2502874 -0.1918835 -0.2400006\n",
       "5  0.1887429  0.2349126  0.2531332 -0.05749007 -0.2211869 -0.1693583 -0.2286415\n",
       "6  0.1887429  0.2349126  0.2531332 -0.57860845 -0.2211869  1.3350119  0.2711608\n",
       "7  0.1887429  0.2349126  0.2531332  0.17894364 -0.1909996 -0.1782122 -0.2594815\n",
       "  PAY_AMT4   PAY_AMT5   PAY_AMT6    default_payment\n",
       "2 -0.3080574 -0.3141309 -0.29337717 1              \n",
       "3 -0.2442256 -0.3141309 -0.18087519 1              \n",
       "4 -0.2442256 -0.2486786 -0.01212223 0              \n",
       "5 -0.2378424 -0.2441624 -0.23712618 0              \n",
       "6  0.2664292 -0.2690343 -0.25518275 0              \n",
       "7 -0.2442256 -0.2486786 -0.24837638 0              "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "head(data_new)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The next task we'll do is to split the data for training and testing as we'll use our test data to evaluate our model. We will now split our dataset into train and test. We'll change it to 0.3. Therefore, 30% of the dataset is reserved for testing while the remaining for training. By default, the dataset will also be shuffled before splitting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "#create a list of random number ranging from 1 to number of rows from actual data \n",
    "#and 70% of the data into training data  \n",
    "\n",
    "data2 = sort(sample(nrow(data_new), nrow(data_new)*.7))\n",
    "\n",
    "#creating training data set by selecting the output row values\n",
    "train <- data_new[data2,]\n",
    "\n",
    "#creating test data set by not selecting the output row values\n",
    "test <- data_new[-data2,]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us print the dimensions of all these variables using the dim method. You can notice the 70-30% split."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<ol class=list-inline>\n",
       "\t<li>21000</li>\n",
       "\t<li>18</li>\n",
       "</ol>\n"
      ],
      "text/latex": [
       "\\begin{enumerate*}\n",
       "\\item 21000\n",
       "\\item 18\n",
       "\\end{enumerate*}\n"
      ],
      "text/markdown": [
       "1. 21000\n",
       "2. 18\n",
       "\n",
       "\n"
      ],
      "text/plain": [
       "[1] 21000    18"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<ol class=list-inline>\n",
       "\t<li>9000</li>\n",
       "\t<li>18</li>\n",
       "</ol>\n"
      ],
      "text/latex": [
       "\\begin{enumerate*}\n",
       "\\item 9000\n",
       "\\item 18\n",
       "\\end{enumerate*}\n"
      ],
      "text/markdown": [
       "1. 9000\n",
       "2. 18\n",
       "\n",
       "\n"
      ],
      "text/plain": [
       "[1] 9000   18"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "dim(train)\n",
    "dim(test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6) Model Development\n",
    "\n",
    "We will now move on to our most important step of developing our logistic regression model. We have already fetched our machine learning model in the beginning. Now with a few lines of code we'll first create a logistic regression model which as has been imported from scikit learn's linear model package to our variable  named *model*. \n",
    "\n",
    "Followed by this, we'll train our model using the fit method with X_train and y_train that contain 70% of our dataset. This will be a  binary classification model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "## fit a logistic regression model with the training dataset\n",
    "log.model <- glm(default_payment ~., data = train, family = binomial(link = \"logit\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       "Call:\n",
       "glm(formula = default_payment ~ ., family = binomial(link = \"logit\"), \n",
       "    data = train)\n",
       "\n",
       "Deviance Residuals: \n",
       "    Min       1Q   Median       3Q      Max  \n",
       "-3.1171  -0.6998  -0.5473  -0.2946   3.4915  \n",
       "\n",
       "Coefficients:\n",
       "             Estimate Std. Error z value Pr(>|z|)    \n",
       "(Intercept) -1.465097   0.019825 -73.900  < 2e-16 ***\n",
       "LIMIT_BAL   -0.083475   0.023905  -3.492 0.000480 ***\n",
       "SEX         -0.082986   0.017717  -4.684 2.81e-06 ***\n",
       "EDUCATION   -0.059851   0.019178  -3.121 0.001803 ** \n",
       "MARRIAGE    -0.107322   0.018350  -5.849 4.95e-09 ***\n",
       "PAY_0        0.661918   0.023605  28.041  < 2e-16 ***\n",
       "PAY_2        0.069704   0.028842   2.417 0.015660 *  \n",
       "PAY_3        0.090691   0.031982   2.836 0.004573 ** \n",
       "PAY_4        0.074336   0.034612   2.148 0.031738 *  \n",
       "PAY_5        0.018469   0.036430   0.507 0.612178    \n",
       "PAY_6        0.006314   0.030235   0.209 0.834584    \n",
       "BILL_AMT1   -0.123582   0.023558  -5.246 1.56e-07 ***\n",
       "PAY_AMT1    -0.136745   0.037549  -3.642 0.000271 ***\n",
       "PAY_AMT2    -0.246634   0.056432  -4.370 1.24e-05 ***\n",
       "PAY_AMT3    -0.014662   0.028012  -0.523 0.600677    \n",
       "PAY_AMT4    -0.087782   0.031484  -2.788 0.005300 ** \n",
       "PAY_AMT5    -0.084533   0.030917  -2.734 0.006254 ** \n",
       "PAY_AMT6    -0.027355   0.025707  -1.064 0.287277    \n",
       "---\n",
       "Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n",
       "\n",
       "(Dispersion parameter for binomial family taken to be 1)\n",
       "\n",
       "    Null deviance: 22176  on 20999  degrees of freedom\n",
       "Residual deviance: 19535  on 20982  degrees of freedom\n",
       "AIC: 19571\n",
       "\n",
       "Number of Fisher Scoring iterations: 6\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "summary(log.model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 7) Prediction\n",
    "\n",
    "Below we'll use the predict method to find out the predictions made by our Logistic Regression method. We will first store the predicted results in our y_pred variable and print our the first 10 rows of our test data set. Following this we will print the predicted values of the corresponding rows and the original labels that were stored in y_test for comparision."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<thead><tr><th></th><th scope=col>LIMIT_BAL</th><th scope=col>SEX</th><th scope=col>EDUCATION</th><th scope=col>MARRIAGE</th><th scope=col>PAY_0</th><th scope=col>PAY_2</th><th scope=col>PAY_3</th><th scope=col>PAY_4</th><th scope=col>PAY_5</th><th scope=col>PAY_6</th><th scope=col>BILL_AMT1</th><th scope=col>PAY_AMT1</th><th scope=col>PAY_AMT2</th><th scope=col>PAY_AMT3</th><th scope=col>PAY_AMT4</th><th scope=col>PAY_AMT5</th><th scope=col>PAY_AMT6</th><th scope=col>default_payment</th></tr></thead>\n",
       "<tbody>\n",
       "\t<tr><th scope=row>5</th><td>-0.9054832 </td><td> 0.8101472 </td><td> 0.2118664 </td><td>-1.0687794 </td><td> 0.01486028</td><td> 0.1117342 </td><td> 0.1388625 </td><td> 0.1887429 </td><td> 0.2349126 </td><td> 0.2531332 </td><td>-0.05749007</td><td>-0.2211869 </td><td>-0.16935834</td><td>-0.22864147</td><td>-0.23784239</td><td>-0.24416239</td><td>-0.2371262 </td><td>0          </td></tr>\n",
       "\t<tr><th scope=row>10</th><td>-0.2118290 </td><td> 0.8101472 </td><td> 1.5550597 </td><td>-1.0687794 </td><td> 0.01486028</td><td> 0.1117342 </td><td> 1.8098911 </td><td> 0.1887429 </td><td> 0.2349126 </td><td> 0.2531332 </td><td>-0.54237610</td><td>-0.1409492 </td><td>-0.25698524</td><td>-0.27226058</td><td>-0.24422557</td><td>-0.24867860</td><td>-0.2371262 </td><td>0          </td></tr>\n",
       "\t<tr><th scope=row>11</th><td>-1.1367012 </td><td>-1.2343024 </td><td> 1.5550597 </td><td> 0.8491164 </td><td>-1.76481340</td><td>-1.5588500 </td><td>-1.5321662 </td><td>-1.5219182 </td><td>-0.6475540 </td><td>-0.6164414 </td><td>-0.69563023</td><td>-0.3419359 </td><td>-0.25698524</td><td>-0.29679633</td><td> 0.52220348</td><td>-0.24069342</td><td>-0.2933772 </td><td>0          </td></tr>\n",
       "\t<tr><th scope=row>12</th><td> 0.2506070 </td><td> 0.8101472 </td><td> 1.5550597 </td><td> 0.8491164 </td><td> 0.01486028</td><td> 0.1117342 </td><td> 1.8098911 </td><td> 0.1887429 </td><td> 0.2349126 </td><td>-0.6164414 </td><td>-0.54525513</td><td>-0.2027123 </td><td>-0.25646442</td><td>-0.29395654</td><td>-0.28890787</td><td>-0.06947024</td><td>-0.2896646 </td><td>0          </td></tr>\n",
       "\t<tr><th scope=row>14</th><td> 3.5647323 </td><td> 0.8101472 </td><td> 0.2118664 </td><td> 0.8491164 </td><td>-0.87497656</td><td> 0.1117342 </td><td>-0.6966518 </td><td>-0.6665876 </td><td>-0.6475540 </td><td>-0.6164414 </td><td>-0.53080565</td><td>-0.2815614 </td><td> 0.02512216</td><td> 0.07237583</td><td> 0.10684961</td><td>-0.12628283</td><td>-0.2933772 </td><td>0          </td></tr>\n",
       "\t<tr><th scope=row>15</th><td>-0.7513378 </td><td>-1.2343024 </td><td> 0.2118664 </td><td> 0.8491164 </td><td> 0.90469711</td><td> 1.7823185 </td><td> 1.8098911 </td><td> 0.1887429 </td><td> 0.2349126 </td><td> 1.9922823 </td><td> 0.19798328</td><td>-0.1487375 </td><td>-0.25698524</td><td>-0.12640918</td><td>-0.11656187</td><td>-0.21595246</td><td>-0.2933772 </td><td>1          </td></tr>\n",
       "\t<tr><th scope=row>19</th><td> 1.1754792 </td><td>-1.2343024 </td><td>-1.1313270 </td><td>-1.0687794 </td><td> 0.01486028</td><td> 0.1117342 </td><td> 0.1388625 </td><td>-0.6665876 </td><td>-0.6475540 </td><td>-0.6164414 </td><td> 2.74407969</td><td> 0.2834233 </td><td> 0.17702615</td><td> 4.01627042</td><td> 0.96857963</td><td>12.48827039</td><td> 2.5191722 </td><td>0          </td></tr>\n",
       "\t<tr><th scope=row>22</th><td>-0.2889017 </td><td> 0.8101472 </td><td> 1.5550597 </td><td> 0.8491164 </td><td> 0.01486028</td><td> 0.1117342 </td><td> 0.1388625 </td><td> 0.1887429 </td><td> 0.2349126 </td><td>-0.6164414 </td><td>-0.17471556</td><td>-0.1608124 </td><td>-0.19027769</td><td>-0.24000061</td><td>-0.18039372</td><td>-0.25326026</td><td> 1.6058812 </td><td>0          </td></tr>\n",
       "\t<tr><th scope=row>23</th><td>-0.3659744 </td><td> 0.8101472 </td><td> 0.2118664 </td><td>-1.0687794 </td><td>-0.87497656</td><td>-0.7235579 </td><td>-0.6966518 </td><td>-0.6665876 </td><td>-0.6475540 </td><td>-0.6164414 </td><td>-0.69133885</td><td>-0.3228576 </td><td>-0.24327048</td><td>-0.29679633</td><td>-0.26771570</td><td>-0.29344796</td><td>-0.2933772 </td><td>1          </td></tr>\n",
       "\t<tr><th scope=row>24</th><td>-0.7513378 </td><td> 0.8101472 </td><td> 0.2118664 </td><td> 0.8491164 </td><td> 1.79453395</td><td> 0.1117342 </td><td> 0.1388625 </td><td> 1.8994041 </td><td> 1.9998457 </td><td> 1.9922823 </td><td>-0.13765482</td><td>-0.2207643 </td><td>-0.10152236</td><td>-0.29679633</td><td>-0.07819893</td><td>-0.31413088</td><td>-0.1910004 </td><td>1          </td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "\\begin{tabular}{r|llllllllllllllllll}\n",
       "  & LIMIT\\_BAL & SEX & EDUCATION & MARRIAGE & PAY\\_0 & PAY\\_2 & PAY\\_3 & PAY\\_4 & PAY\\_5 & PAY\\_6 & BILL\\_AMT1 & PAY\\_AMT1 & PAY\\_AMT2 & PAY\\_AMT3 & PAY\\_AMT4 & PAY\\_AMT5 & PAY\\_AMT6 & default\\_payment\\\\\n",
       "\\hline\n",
       "\t5 & -0.9054832  &  0.8101472  &  0.2118664  & -1.0687794  &  0.01486028 &  0.1117342  &  0.1388625  &  0.1887429  &  0.2349126  &  0.2531332  & -0.05749007 & -0.2211869  & -0.16935834 & -0.22864147 & -0.23784239 & -0.24416239 & -0.2371262  & 0          \\\\\n",
       "\t10 & -0.2118290  &  0.8101472  &  1.5550597  & -1.0687794  &  0.01486028 &  0.1117342  &  1.8098911  &  0.1887429  &  0.2349126  &  0.2531332  & -0.54237610 & -0.1409492  & -0.25698524 & -0.27226058 & -0.24422557 & -0.24867860 & -0.2371262  & 0          \\\\\n",
       "\t11 & -1.1367012  & -1.2343024  &  1.5550597  &  0.8491164  & -1.76481340 & -1.5588500  & -1.5321662  & -1.5219182  & -0.6475540  & -0.6164414  & -0.69563023 & -0.3419359  & -0.25698524 & -0.29679633 &  0.52220348 & -0.24069342 & -0.2933772  & 0          \\\\\n",
       "\t12 &  0.2506070  &  0.8101472  &  1.5550597  &  0.8491164  &  0.01486028 &  0.1117342  &  1.8098911  &  0.1887429  &  0.2349126  & -0.6164414  & -0.54525513 & -0.2027123  & -0.25646442 & -0.29395654 & -0.28890787 & -0.06947024 & -0.2896646  & 0          \\\\\n",
       "\t14 &  3.5647323  &  0.8101472  &  0.2118664  &  0.8491164  & -0.87497656 &  0.1117342  & -0.6966518  & -0.6665876  & -0.6475540  & -0.6164414  & -0.53080565 & -0.2815614  &  0.02512216 &  0.07237583 &  0.10684961 & -0.12628283 & -0.2933772  & 0          \\\\\n",
       "\t15 & -0.7513378  & -1.2343024  &  0.2118664  &  0.8491164  &  0.90469711 &  1.7823185  &  1.8098911  &  0.1887429  &  0.2349126  &  1.9922823  &  0.19798328 & -0.1487375  & -0.25698524 & -0.12640918 & -0.11656187 & -0.21595246 & -0.2933772  & 1          \\\\\n",
       "\t19 &  1.1754792  & -1.2343024  & -1.1313270  & -1.0687794  &  0.01486028 &  0.1117342  &  0.1388625  & -0.6665876  & -0.6475540  & -0.6164414  &  2.74407969 &  0.2834233  &  0.17702615 &  4.01627042 &  0.96857963 & 12.48827039 &  2.5191722  & 0          \\\\\n",
       "\t22 & -0.2889017  &  0.8101472  &  1.5550597  &  0.8491164  &  0.01486028 &  0.1117342  &  0.1388625  &  0.1887429  &  0.2349126  & -0.6164414  & -0.17471556 & -0.1608124  & -0.19027769 & -0.24000061 & -0.18039372 & -0.25326026 &  1.6058812  & 0          \\\\\n",
       "\t23 & -0.3659744  &  0.8101472  &  0.2118664  & -1.0687794  & -0.87497656 & -0.7235579  & -0.6966518  & -0.6665876  & -0.6475540  & -0.6164414  & -0.69133885 & -0.3228576  & -0.24327048 & -0.29679633 & -0.26771570 & -0.29344796 & -0.2933772  & 1          \\\\\n",
       "\t24 & -0.7513378  &  0.8101472  &  0.2118664  &  0.8491164  &  1.79453395 &  0.1117342  &  0.1388625  &  1.8994041  &  1.9998457  &  1.9922823  & -0.13765482 & -0.2207643  & -0.10152236 & -0.29679633 & -0.07819893 & -0.31413088 & -0.1910004  & 1          \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "| <!--/--> | LIMIT_BAL | SEX | EDUCATION | MARRIAGE | PAY_0 | PAY_2 | PAY_3 | PAY_4 | PAY_5 | PAY_6 | BILL_AMT1 | PAY_AMT1 | PAY_AMT2 | PAY_AMT3 | PAY_AMT4 | PAY_AMT5 | PAY_AMT6 | default_payment |\n",
       "|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|\n",
       "| 5 | -0.9054832  |  0.8101472  |  0.2118664  | -1.0687794  |  0.01486028 |  0.1117342  |  0.1388625  |  0.1887429  |  0.2349126  |  0.2531332  | -0.05749007 | -0.2211869  | -0.16935834 | -0.22864147 | -0.23784239 | -0.24416239 | -0.2371262  | 0           |\n",
       "| 10 | -0.2118290  |  0.8101472  |  1.5550597  | -1.0687794  |  0.01486028 |  0.1117342  |  1.8098911  |  0.1887429  |  0.2349126  |  0.2531332  | -0.54237610 | -0.1409492  | -0.25698524 | -0.27226058 | -0.24422557 | -0.24867860 | -0.2371262  | 0           |\n",
       "| 11 | -1.1367012  | -1.2343024  |  1.5550597  |  0.8491164  | -1.76481340 | -1.5588500  | -1.5321662  | -1.5219182  | -0.6475540  | -0.6164414  | -0.69563023 | -0.3419359  | -0.25698524 | -0.29679633 |  0.52220348 | -0.24069342 | -0.2933772  | 0           |\n",
       "| 12 |  0.2506070  |  0.8101472  |  1.5550597  |  0.8491164  |  0.01486028 |  0.1117342  |  1.8098911  |  0.1887429  |  0.2349126  | -0.6164414  | -0.54525513 | -0.2027123  | -0.25646442 | -0.29395654 | -0.28890787 | -0.06947024 | -0.2896646  | 0           |\n",
       "| 14 |  3.5647323  |  0.8101472  |  0.2118664  |  0.8491164  | -0.87497656 |  0.1117342  | -0.6966518  | -0.6665876  | -0.6475540  | -0.6164414  | -0.53080565 | -0.2815614  |  0.02512216 |  0.07237583 |  0.10684961 | -0.12628283 | -0.2933772  | 0           |\n",
       "| 15 | -0.7513378  | -1.2343024  |  0.2118664  |  0.8491164  |  0.90469711 |  1.7823185  |  1.8098911  |  0.1887429  |  0.2349126  |  1.9922823  |  0.19798328 | -0.1487375  | -0.25698524 | -0.12640918 | -0.11656187 | -0.21595246 | -0.2933772  | 1           |\n",
       "| 19 |  1.1754792  | -1.2343024  | -1.1313270  | -1.0687794  |  0.01486028 |  0.1117342  |  0.1388625  | -0.6665876  | -0.6475540  | -0.6164414  |  2.74407969 |  0.2834233  |  0.17702615 |  4.01627042 |  0.96857963 | 12.48827039 |  2.5191722  | 0           |\n",
       "| 22 | -0.2889017  |  0.8101472  |  1.5550597  |  0.8491164  |  0.01486028 |  0.1117342  |  0.1388625  |  0.1887429  |  0.2349126  | -0.6164414  | -0.17471556 | -0.1608124  | -0.19027769 | -0.24000061 | -0.18039372 | -0.25326026 |  1.6058812  | 0           |\n",
       "| 23 | -0.3659744  |  0.8101472  |  0.2118664  | -1.0687794  | -0.87497656 | -0.7235579  | -0.6966518  | -0.6665876  | -0.6475540  | -0.6164414  | -0.69133885 | -0.3228576  | -0.24327048 | -0.29679633 | -0.26771570 | -0.29344796 | -0.2933772  | 1           |\n",
       "| 24 | -0.7513378  |  0.8101472  |  0.2118664  |  0.8491164  |  1.79453395 |  0.1117342  |  0.1388625  |  1.8994041  |  1.9998457  |  1.9922823  | -0.13765482 | -0.2207643  | -0.10152236 | -0.29679633 | -0.07819893 | -0.31413088 | -0.1910004  | 1           |\n",
       "\n"
      ],
      "text/plain": [
       "   LIMIT_BAL  SEX        EDUCATION  MARRIAGE   PAY_0       PAY_2     \n",
       "5  -0.9054832  0.8101472  0.2118664 -1.0687794  0.01486028  0.1117342\n",
       "10 -0.2118290  0.8101472  1.5550597 -1.0687794  0.01486028  0.1117342\n",
       "11 -1.1367012 -1.2343024  1.5550597  0.8491164 -1.76481340 -1.5588500\n",
       "12  0.2506070  0.8101472  1.5550597  0.8491164  0.01486028  0.1117342\n",
       "14  3.5647323  0.8101472  0.2118664  0.8491164 -0.87497656  0.1117342\n",
       "15 -0.7513378 -1.2343024  0.2118664  0.8491164  0.90469711  1.7823185\n",
       "19  1.1754792 -1.2343024 -1.1313270 -1.0687794  0.01486028  0.1117342\n",
       "22 -0.2889017  0.8101472  1.5550597  0.8491164  0.01486028  0.1117342\n",
       "23 -0.3659744  0.8101472  0.2118664 -1.0687794 -0.87497656 -0.7235579\n",
       "24 -0.7513378  0.8101472  0.2118664  0.8491164  1.79453395  0.1117342\n",
       "   PAY_3      PAY_4      PAY_5      PAY_6      BILL_AMT1   PAY_AMT1  \n",
       "5   0.1388625  0.1887429  0.2349126  0.2531332 -0.05749007 -0.2211869\n",
       "10  1.8098911  0.1887429  0.2349126  0.2531332 -0.54237610 -0.1409492\n",
       "11 -1.5321662 -1.5219182 -0.6475540 -0.6164414 -0.69563023 -0.3419359\n",
       "12  1.8098911  0.1887429  0.2349126 -0.6164414 -0.54525513 -0.2027123\n",
       "14 -0.6966518 -0.6665876 -0.6475540 -0.6164414 -0.53080565 -0.2815614\n",
       "15  1.8098911  0.1887429  0.2349126  1.9922823  0.19798328 -0.1487375\n",
       "19  0.1388625 -0.6665876 -0.6475540 -0.6164414  2.74407969  0.2834233\n",
       "22  0.1388625  0.1887429  0.2349126 -0.6164414 -0.17471556 -0.1608124\n",
       "23 -0.6966518 -0.6665876 -0.6475540 -0.6164414 -0.69133885 -0.3228576\n",
       "24  0.1388625  1.8994041  1.9998457  1.9922823 -0.13765482 -0.2207643\n",
       "   PAY_AMT2    PAY_AMT3    PAY_AMT4    PAY_AMT5    PAY_AMT6   default_payment\n",
       "5  -0.16935834 -0.22864147 -0.23784239 -0.24416239 -0.2371262 0              \n",
       "10 -0.25698524 -0.27226058 -0.24422557 -0.24867860 -0.2371262 0              \n",
       "11 -0.25698524 -0.29679633  0.52220348 -0.24069342 -0.2933772 0              \n",
       "12 -0.25646442 -0.29395654 -0.28890787 -0.06947024 -0.2896646 0              \n",
       "14  0.02512216  0.07237583  0.10684961 -0.12628283 -0.2933772 0              \n",
       "15 -0.25698524 -0.12640918 -0.11656187 -0.21595246 -0.2933772 1              \n",
       "19  0.17702615  4.01627042  0.96857963 12.48827039  2.5191722 0              \n",
       "22 -0.19027769 -0.24000061 -0.18039372 -0.25326026  1.6058812 0              \n",
       "23 -0.24327048 -0.29679633 -0.26771570 -0.29344796 -0.2933772 1              \n",
       "24 -0.10152236 -0.29679633 -0.07819893 -0.31413088 -0.1910004 1              "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "test[1:10,]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<dl class=dl-horizontal>\n",
       "\t<dt>5</dt>\n",
       "\t\t<dd>0.236122461087491</dd>\n",
       "\t<dt>10</dt>\n",
       "\t\t<dd>0.251870688502323</dd>\n",
       "\t<dt>11</dt>\n",
       "\t\t<dd>0.0565856643656278</dd>\n",
       "\t<dt>12</dt>\n",
       "\t\t<dd>0.207611552792466</dd>\n",
       "\t<dt>14</dt>\n",
       "\t\t<dd>0.073952958431911</dd>\n",
       "\t<dt>15</dt>\n",
       "\t\t<dd>0.404203452365701</dd>\n",
       "\t<dt>19</dt>\n",
       "\t\t<dd>0.047264262658606</dd>\n",
       "\t<dt>22</dt>\n",
       "\t\t<dd>0.173593005328485</dd>\n",
       "\t<dt>23</dt>\n",
       "\t\t<dd>0.129243184708745</dd>\n",
       "\t<dt>24</dt>\n",
       "\t\t<dd>0.485188766936668</dd>\n",
       "</dl>\n"
      ],
      "text/latex": [
       "\\begin{description*}\n",
       "\\item[5] 0.236122461087491\n",
       "\\item[10] 0.251870688502323\n",
       "\\item[11] 0.0565856643656278\n",
       "\\item[12] 0.207611552792466\n",
       "\\item[14] 0.073952958431911\n",
       "\\item[15] 0.404203452365701\n",
       "\\item[19] 0.047264262658606\n",
       "\\item[22] 0.173593005328485\n",
       "\\item[23] 0.129243184708745\n",
       "\\item[24] 0.485188766936668\n",
       "\\end{description*}\n"
      ],
      "text/markdown": [
       "5\n",
       ":   0.23612246108749110\n",
       ":   0.25187068850232311\n",
       ":   0.056585664365627812\n",
       ":   0.20761155279246614\n",
       ":   0.07395295843191115\n",
       ":   0.40420345236570119\n",
       ":   0.04726426265860622\n",
       ":   0.17359300532848523\n",
       ":   0.12924318470874524\n",
       ":   0.485188766936668\n",
       "\n"
      ],
      "text/plain": [
       "         5         10         11         12         14         15         19 \n",
       "0.23612246 0.25187069 0.05658566 0.20761155 0.07395296 0.40420345 0.04726426 \n",
       "        22         23         24 \n",
       "0.17359301 0.12924318 0.48518877 "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "## to predict using logistic regression model, probablilities obtained\n",
    "log.predictions <- predict(log.model, test, type=\"response\")\n",
    "\n",
    "## Look at probability output\n",
    "head(log.predictions, 10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below we are going to assign our labels with decision rule that if the prediction is greater than 0.5, assign it 1 else 0."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<dl class=dl-horizontal>\n",
       "\t<dt>5</dt>\n",
       "\t\t<dd>0</dd>\n",
       "\t<dt>10</dt>\n",
       "\t\t<dd>0</dd>\n",
       "\t<dt>11</dt>\n",
       "\t\t<dd>0</dd>\n",
       "\t<dt>12</dt>\n",
       "\t\t<dd>0</dd>\n",
       "\t<dt>14</dt>\n",
       "\t\t<dd>0</dd>\n",
       "\t<dt>15</dt>\n",
       "\t\t<dd>0</dd>\n",
       "\t<dt>19</dt>\n",
       "\t\t<dd>0</dd>\n",
       "\t<dt>22</dt>\n",
       "\t\t<dd>0</dd>\n",
       "\t<dt>23</dt>\n",
       "\t\t<dd>0</dd>\n",
       "\t<dt>24</dt>\n",
       "\t\t<dd>0</dd>\n",
       "</dl>\n"
      ],
      "text/latex": [
       "\\begin{description*}\n",
       "\\item[5] 0\n",
       "\\item[10] 0\n",
       "\\item[11] 0\n",
       "\\item[12] 0\n",
       "\\item[14] 0\n",
       "\\item[15] 0\n",
       "\\item[19] 0\n",
       "\\item[22] 0\n",
       "\\item[23] 0\n",
       "\\item[24] 0\n",
       "\\end{description*}\n"
      ],
      "text/markdown": [
       "5\n",
       ":   010\n",
       ":   011\n",
       ":   012\n",
       ":   014\n",
       ":   015\n",
       ":   019\n",
       ":   022\n",
       ":   023\n",
       ":   024\n",
       ":   0\n",
       "\n"
      ],
      "text/plain": [
       " 5 10 11 12 14 15 19 22 23 24 \n",
       " 0  0  0  0  0  0  0  0  0  0 "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "log.prediction.rd <- ifelse(log.predictions > 0.5, 1, 0)\n",
    "head(log.prediction.rd, 10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 8) Evaluation\n",
    "\n",
    "We'll now discuss a few evaluation metrics to measure the performance of our machine learning model here. This part has significant relevance since it will allow us to understand the most important characteristics that led to our model development. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will output the confusion matrix. It is a handy presentation of the accuracy of a model with two or more classes.\n",
    "\n",
    "The table presents predictions on the x-axis and accuracy outcomes on the y-axis. The cells of the table are the number of predictions made by a machine learning algorithm.\n",
    "\n",
    "According to an [article](http://www2.cs.uregina.ca/~dbd/cs831/notes/confusion_matrix/confusion_matrix.html) the entries in the confusion matrix have the following meaning in the context of our study:\n",
    "\n",
    "[[a  b]<br/>\n",
    " [c  d]]\n",
    "\n",
    "* a is the number of correct predictions that an instance is negative,\n",
    "* b is the number of incorrect predictions that an instance is positive,\n",
    "* c is the number of incorrect of predictions that an instance is negative, and\n",
    "* d is the number of correct predictions that an instance is positive.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "                 \n",
       "log.prediction.rd    0    1\n",
       "                0 6832 1517\n",
       "                1  170  481"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "table(log.prediction.rd, test[,18])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll write a simple function to print the accuracy below"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "0.812555555555556"
      ],
      "text/latex": [
       "0.812555555555556"
      ],
      "text/markdown": [
       "0.812555555555556"
      ],
      "text/plain": [
       "[1] 0.8125556"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "accuracy <- table(log.prediction.rd, test[,18])\n",
    "sum(diag(accuracy))/sum(accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "\n",
    "This tutorial has given you a brief and concise overview of Logistic Regression algorithm and all the steps involved in acheiving better results from our model. This notebook has also highlighted a few methods related to Exploratory Data Analysis, Pre-processing and Evaluation, however, there are several other methods that we would encourage to explore on our [blog](https://blog.datasciencedojo.com/) or [video tutorials](https://tutorials.datasciencedojo.com/). \n",
    "\n",
    "If you want to take a deeper dive into the several data science techniques. Join our 5-day hands-on [data science bootcamp](https://datasciencedojo.com/data-science-bootcamp/) preferred by working professionals, we cover the following topics: \n",
    "* Fundamentals of Data Mining <br/>\n",
    "* Machine Learning Fundamentals <br/>\n",
    "* Introduction to R <br/>\n",
    "* Introduction to Azure Machine Learning Studio <br/>\n",
    "* Data Exploration, Visualization, and Feature Engineering <br/>\n",
    "* Decision Tree Learning <br/>\n",
    "* Ensemble Methods: Bagging, Boosting, and Random Forest <br/>\n",
    "* Regression: Cost Functions, Gradient Descent, Regularization <br/>\n",
    "* Unsupervised Learning <br/>\n",
    "* Recommendation Systems <br/>\n",
    "* Metrics and Methods for Evaluating Predictive Models <br/>\n",
    "* Introduction to Online Experimentation and A/B Testing <br/>\n",
    "* Fundamentals of Big Data Engineering <br/>\n",
    "* Hadoop and Hive <br/>\n",
    "* Message Queues and Real-time Analytics <br/>\n",
    "* NoSQL Databases and HBase <br/>\n",
    "* Hack Project: Creating a Real-time IoT Pipeline <br/>\n",
    "* Naive Bayes <br/>\n",
    "* Logistic Regression <br/>\n",
    "* Times Series Forecasting \n",
    "\n",
    "About The Author\n",
    "Rahim Rasool is an Associate Data Scientist at Data Science Dojo (DSD) where he helps create learning material for DSD’s [data science bootcamp](https://datasciencedojo.com/data-science-bootcamp/). He holds a bachelor’s in electrical engineering from National University of Sciences and Technology. He possesses great interest in machine learning, astronomy and history."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}